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Abstract 



An investigation is made into the problem of constructing 
a model of the appearance to an optical input device of scenes 
consisting of plane-faced geometric solids. The goal is to 
study algorithms which find the real straight edges in the 
scenes, taking into account smooth variations in intensity over 
faces of the solids, blurring of edges and noise. A general 
mathematical analysis is made of optimal methods for identify- 
ing the edge lines in figures, given a raster of intensities 
covering the entire field of view. There is given in addition 
a suboptimal statistical decision procedure, based on the model, 
for the identification of a line within a narrow band on the 
field of view given an array of intensities from within the 
band. A computer program has been written and extensively 
tested which implements this procedure and extracts lines 
from real scenes. Other programs were written which judge 
the completeness of extracted sets of lines, and propose and 
test for additional lines which had escaped initial detection. 
The performance of these programs is discussed in relation to 
the theory derived from the model, and with regard to their 
use of global information in detecting and proposing lines. 
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CHAPTER 1 

TOWARD A THEORY OF THE OPTIMAL USE OF INTENSITY 

INFORMATION IN THE DETECTION OF LINES IN A 

VISUAL FIELD 

I. INTRODUCTION AND DEFINITIONS 

We are interested in studying procedures which identify 
the images of straight lines in some two-dimensional (picture-like) 
projection of an array of real objects, given some large set of 
intensity values from points on the two-dimensional projection 
plane. We shall confine our attention to arrays of plane -faced 
solids. Applying a line -detection procedure to the image of an 
array of such objects yields a "description" of the objects which 
is, in effect, a line drawing of the objects from a particular 
point of view. Such a description is complete in the sense that it 
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contains all information obtainable from that point of view which is 
relevant to determining the shape, location in space, and so forth 
of the objects. Information of the latter sort, which may be 
regarded as constituting a more detailed description of the array 
of objects, may be derived from one or several "line drawings" 
of a particular array of objects, by methods surveyed in the last 
chapter of this paper. 

In the first chapters of this paper, we shall investigate the 
problem of constructing line detection procedures which make 
optimal use of the set of intensity values to which they are 
applied, by being maximally accurate, in various senses, about 
the lines they claim to exist. Such investigations require a 
formalism, definitions and so forth, which will presently be 
developed in this section. In the meantime we shall informally 
discuss several considerations relevant to the development of 
such a theory, with the idea of motivating some of the formal 
presentation of the rest of the chapter. 

First, it appears that a theory of optimal edge finding 
procedures would be intractably difficult if we were to consider 
procedures which decide on the locations at which to measure 
each successive intensity value. This leads into the realm of 
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sequential decision procedures; and we should not attempt this 
until we first have a reasonable understanding of the simpler 
non- sequential case. One may get an idea of the inherent 
complexity of optimal sequential decision procedures by 
consulting, e. g. , Bellman (Bellman). For the moment, then, 
we shall assume that the intensities are obtained from the two- 
dimensional projection of the array of objects in a predetermined 
pattern, such as at the vertices of a grid of squares, and that the 
line finding procedure acts on exactly these intensities. We 
shall return to the problem of sequential analysis of scenes in 
chapter four, where the matter is approached at the heuristic 
level. 

A second assumption to be made concerning the set of 
intensities upon which a line finding procedure acts concerns 
the relation between the actual intensity at a point in the two 
dimensional view of the scene and the intensity available to 
the line -finding procedure, which would be obtained by a real 
optical input device. In general such a device is subject to 
various forms of optical blurring and distortion, as well as to 
random noise. If there is nothing known about the noise, then 
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the set of intensity values obtained by such a device contains no 
information about the array of objects at hand, in particular 
about the locations of the projections of their edge lines. It is 
a reasonable assumption that the obtained values of intensity at 
any point be normally distributed about the actual intensity, and 
that the standard deviation of such a distribution be a function 
of the intensity at the point. It is further reasonable to assume 
that any spatial distortion of the intensities be accountable for 
principally as some sort of blurring phenomenon. These forms 
of distortion and noise are present in any real optical or electro- 
optical system; and certain devices, such as an image -dissector, 
may be designed so that these are the principal effects. 

A third restriction involves the relation between the size 
of the objects in the scenes under consideration, or more 
properly the size of the images of these objects in a two 
dimensional projection, and the spatial density of points at which 
intensities are measured. If the intensities are obtained in a 
simple grid pattern with a spacing of one unit on the image plane, 
it would clearly be impossible to detect edge lines whose images, 
for example, were shorter than one unit. In judging the 
optimality of an edge detector, it would surely be unfair to 
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penalize it for not detecting so short a line. A solution to the 
problem might be to consider only arrays of objects whose 
images are larger than a certain size relative to the spacing of 
the points at which intensities are measured. An alternative 
approach, which will actually be employed, is as follows: Choose 
a suitable geometry of points in the image space at which to 
measure intensities. Then define a simple predicate on exact 
values of intensity which would be obtained at these points in the 
absence of all noise, distortion and imperfections in the objects 
being examined. Then use this criterion for the existence of a 
line to judge the performance of a line detection procedure which 
measures intensities at the same points, but with a real optical 
input device. By this means, one is essentially determining how 
well the procedure "sees through" the distortion and noise; and does 
not penalize it for not "seeing" lines that it could not possibly 
detect from intensities measured at points of the given grid 
even in the absence of noise. 

The concepts so far discussed may be summarized in the 
following definitions. 
Definition: an intensity function , I(x, y), is a real-valued 

function whose values and derivatives are defined 
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on a square in 2 -space, except for a set of 

measure 0. 
Definition: a description space, D, is an arbitrary set of 

elements jdj^}. 
Definition: a sampling set , S, is a finite set of points 

{(x.,y.), ... [. 
Definition: a noise-free sample , I(S) , of an intensity function 

I(x, y), is the set of values of I(x., y0 for the 

points (Xj^, y.-) of S. 
Definition: a noisy sample , J(S), is defined as above, but with 

the values of intensity modified by some distortion 

and noise functions. 
Definition: an identification function , F(I), is a mapping from 

a set of intensity functions onto some description 

space. 

F: |l} ->D 
Definition: a noise -free sample identification function, G(I(S)), 

is a mapping from a set of noise -free samples onto 

a description space. 

G: {l(S)[ ->D 
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Definition: a noisy-sample identification function , H(J(S)) , 
is a mapping from a set of noisy samples to a 
description space. 
H: {J(S)| ->D 
Definition: a sample identification function G, together with a 

space of noise-free samples {l(S)| and a description 
space D are said to be descriptively complete with 
respect to an identification function F if 
F(I)=d- <^G(I(S))=d. 
This definition describes the situation where all that one wishes to 
say about a scene may be inferred from the values of intensity at 
a predetermined finite set of locations on the visual field. 
Definition: an intensity function I*(x, y) on a domain A is said 
to be equivalent under the blurring function 
B(x, y) to a function I(x, y) if: 



I*(x, y)= / I(u, v)B(u-x, v-y)du dv 



Definition: an intensity function I**(x, y) is said to be 

Gaussian-noise modulated with respect to an 
intensity function I*(x, y) if successive values of 
I**(x_, y ) distribute normally about I*(xp., y^) with 
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a standard deviation which is dependent solely on 
the value of I*(X(,, y^)- 
Definition: a transducer which goes from an intensity signal 
I(x, y) to a Gaussian-noise modulated function 
I**(x, y) of a function I*(x, y) which is equivalent 

under blurring to I(x, y), is called a transducer 

subject to fundamental distortion. 
Definition: a scene system with identifiers consists of: 

]l.(x, y)[, a set of intensity functions on a common 

domain; 

|(x.,y.)[ = S, a sampling set; 

|l-(S)}, a set of noise-free samples, one for each 

intensity function I^; 

I J-(S)|, a set of noisy samples, an infinite number 

of J.(S) for each I.(S). The relation between a set 

of Ji(S) and the corresponding Ii(S) is entirely 

accountable for by fundamental distortion. 

{dj^l = D, a description space; 

F: |li(x, y)| — >D, an identification function; 

G: |l.(S)} — >D, a noise -free sample identification 

function; 

H: |j.(S)} — >D, a noisy-sample identification function. 
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Finally, for completeness: 

Definition: a scene system with identifiers is said to be 

descriptively complete if the relevant parts conform 
to the descriptive completeness criterion above. 
The various restrictions set forth in the earlier parts of 

this section may be simply summarized by saying that we will 

concern ourselves exclusively with scene systems with identifiers 

which are descriptively complete. 

II. COMPLETE SYSTEMS OF STRAIGHT -LINE SCENES 

A criterion for what constitutes a line in a noise-free 
sample will now be given. This definition will provide a basis 
for a definition of a line in an intensity function which will be 
appropriate for ensuring completeness. 

Prior to making this definition, however, it is necessary 
to digress slightly to consider a plausible restriction on the 
geometry of the sampling set. An important desideratum of any 
line finder is that it not be especially sensitive to lines in a 
particular part of the field of view, or at any particular 
orientation in the field of view. It may be that of two procedures, 
one will find more lines with a given amount of effort, but at the 
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expense of uniformity in this sense. One would still not prefer 
the "better" line finder, on account of its inhomogeneous or 
anisotropis behavior. One thus wishes to guarantee that any line- 
finding procedure that is a candidate for being considered optimal 
is in addition uniform in this sense. A necessary condition seems 
to be to have a sampling set which is distributed uniformlu over 
the entire field of view. Without proving this last assertion, we 
shall henceforth restrict our attention to systems whose sampling 
sets consist of points on all vertices of some grid of squares 
covering the entire field of view. 

A straight line in a noise -free sample over a sampling 
region as described above may be defined in terms of the notions 
of discontinuity and anomaly. We define three such notions 
for, e. g. , a set of colinear intensities ii, ig, 13 and i^, using 
constants K, K' and K": 

1) A discontinuity of intensity occurs at a point between the 
points at which ig and i were obtained if: 

|i2 - isl > Kdii - iai +^3 " H^^- 

2) An anomalous intensit y occurs at the point where ig 

was obtained if: 



|i2 - (i^ - i3)/2| > KV 
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3) A discontinuity in shading occurs at a point between 

the points at which i and i were obtained if: 

^ o 

|(il -i^) -(ig -i^)! >K". 

A line may be defined as two or more such points, all of the 
sanne type and all adjacent and colinear. 

It is now easy to specify a class of systems with identifiers 
which are descriptively complete in their identification of lines in 
a scene. One may simply choose a scene system with a description 
space appropriate for describing lines, and define a noise-free 
sample identification function G in such a way that it appropriately 
identifies all the lines in a noise -free sample according to the above 
definition of a line. One may then define the identification function 
F as follows: 

F(I)=d.<t=:^ G(I(S))=di. 
The completeness follows a fortiori. 

In whatever system we shall henceforth be considering, we 

shall assume that the description space is the power set of the set 

of all lines definable as above on noise -free samples. We shall 

further assume that the noise=free sample identification function 

of the system is as defined in the previous paragraph. The 

choice of an optimal noisy-sannple identification function for this 

type of system is" the subject of the remainder of this paper. 
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III. OPTIMIZATION CRITERIA 

The variables to be considered in discussing optimal 
object recognition algorithms are number of intensities read, 
time taken to read each intensity value, length of computation 
time, and rate of error in identifying lines. We will not be 
concerned with computation time, as this would require more 
knowledge about "mimimal" versions of the algorithms concerned 
than is available today. On the other hand, the number of points 
at which intensities are taken, and the length of time taken in 
getting the intensities can be related in a natural way to the 
accuracy of identifying and locating lines, or other features 
of a scene. Intuitively, there must exist a trade-off between 
the measurement cost of a certain amount of intensity 
information; and accuracy, measured in terms of number 
of lines correctly identified. Optimization may consist in 
minimizing some type of error rate, with a fixed overhead 
in effort. One may, alternatively, wish to set a certain 
error rate criterion, and minimize the amount of effort 
necessary to achieve it. It is easy to see that this latter 
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form of optimization may be reduced to the former: Suppose there 
exists a procedure P(E) which, using E units of effort, minimizes 
the error rate. If it is desired to minimize effort for a fixed 
error rate, one may simply consider all possible instances of 
the procedure P(E) with error rate below the desired one, and 
choose the procedure P(E*) where E* is a minimum among all 
such amounts of effort. This procedure uses the least possible 
amount of effort while guaranteeing the error rate to be below a 
certain value. We may thus without loss of generality restrict our 
attention to procedures which make a minimum of errors for a 
given amiount of overhead. 

Further discussion of error rate requires a more detailed 
discussion of various types of classification errors and the 
probabilities with which they occur. We shall base this discussion 
on the notion of false positive error, the assertion by a line finder 
that a line exists at a particular place on the visual field when no 
such line exists in the scene under consideration, and upon the 
converse notion of true negative error. We will begin with some 
definitions regarding line identification functions. 
Definition: In a system of the type under consideration, for 

some noise-free sample I.(S) and a corresponding 
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Definition: 



Definition: 



noisy pattern J^(S), we call the set of lines 
H(J.(S))-G(I-(S)) the set of false positive errors 

J 

in the identification of the scene from the noisy- 
sample J-{S). 

For similar conditions^ the set G(I^(S))-H(J.(S)) 
is called the set of true negative errors in the 
identification of the scene from the noisy sample 
Jj(S). 

Let A and B be two sets, then the difference 
cardinality, N(A, B), is the cardinality of the 
set A-B. 
It follows immediately that for any J-(S) and corresponding 
Ij^(S), the number of false positive errors is given by: 

N{H(J.(S)), G{I.(S))), 
and the number of true negative errors by: 
N(G(Ii{S)),H(J.(S))). 

With the aid of the function N defined above, we may 
indicate the performance of a noisy sample identification function 
in terms of the number of errors of each type it will make in 
describing any particular noisy sample. Before being able to 
indicate the over-all performance of a noisy sample identification 
function, one must as well be able to specify the relative 
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probabilities of occurrence of the various pairs of noise -free and 
noisy samples. For the present we need only the first of the 
following definitions; however they will all ultimately be necessary. 
Definition: P(I.(S), J.(S)) is the joint probability of occurrence 

of the noise-free sample I.(S) and the noisy sample 

Jj(S). 
Definition: P(J-(S)/lj^(S)) is the conditional probability that the 

noisy sample J-j{S) will occur given the occurrence 

of noise-free sample I|{S). 
Definition: P(I.(S)/ Jj{S)) is defined similarly. 
Definition: P(1^(S)), P(J.(S)) are the a priori probabilities of 

occurrence of the noise -free sample I.(S) and the 

noisy sample J-(S). 
We may now discuss the expected rate of error of a 
classification system of the type under consideration. 
Lemma: The false - positive error rate, or expected number 

of false positive errors made by a complete scene 

system is given by: 



Proof: 



X Pd.CS), J^(S))N(H{J (S)), G(I.{S))) 

Whenever the noisy sample J(S) occurs in 
conjunction with the scene representable by 
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the noise; -f'l'ce sample L(S)j the nianber of 
false positive ei'rox'S is: 



\(IT(J^(S)), G(I^(S))). 



Lemma: 



Phis oecLii'S with probability 

QLD 
rh(^ true negative error rate in a similar situation 



P(I.(S). J (S)) 
1 J 



is gi\'en by: 



^ P(I.(S), J.(S))N((KI^(S)), lI(Jj(S))), 



i> 



th(} pi'oof is similar-. 
Willi the aid of the- following defhutions, we may define two 
additional error -rate measures: 
Definition: iVKA.B), the total number of e rrors is defined by: 

:\1(A„ B)=N(A, B)-N(T5.. A). 
Definition: 0(AJ3), tlie id(;ntity predieate , is defined by: 

0{A, B)-l if A-B, 

0(A, B)m3 ifA+B. 
Lemma: The cor-rec t i-e cognition I'ate, or expected rate at 

whiich a sceiK; is perfectly identified is given by 

the following: 

y P(1,(S), J.{S))0(G(J.(S)).. H(I.(S))), 

-^-^ 1 ,1 J ^ 

h J 

tne proof is as above. 
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Theorem: The total error rate, or expected number of errors 
made by a complete scene system is given by: 
X P(Ii(S), Jj(S))M(G(Jj(S)), H(Ii(S))), 

the proof is as above. 
As a corrolary to the above theorems, we make the 

following definitions. 

Definition: a(J.(S), d.)= J^PCl.iS)/ Jj{S))N(d^, G(I.(S))) 

is the expected rate of false positive errors on 
occasions when the noisy sample J^(S) occurs, 
provided the description d- is assigned to it. 

Definition: b(J.(S), dj)=2]P(Ii(S)/ Jj(S))N(G(Ii(S)), d.) 

is the expected error rate of true negative errors 
under similar conditions. 

IV. OPTIMIZATION PROCEDURES 

With the aid of the definitions in the previous section, we 
are in a position to discuss four optimization procedures. The 
first involves the maximization of the probability of correctly 
recognizing a scene in its entirety. The second involves 
minimizing the sum of the false positive and true negative error 
rates. The last two involve the maximization of one of the error 
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rates with the other error rate held constant. Since these latter 

two procedures are mathematically nearly equivalent, we shall 

consider only the case of holding the false positive error rate 

constant, and minimizing the true negative error rate. 

Theorem: The scene recognition procedure which maximizes 

the rate of correct recognition of scenes in their 

entirety is given by the function H(J^)=d, such that 

2]0(Gai(S)h dj^)P(I.(S), Jj(S)) 

i 
is maximized. 

Proof: If description d, is assigned to the noisy sample 

J.(S), then a perfect recognition of the scene will 

occur whenever L(S), the corresponding state of the 

real world, has the property that G(Ij^(S))=dj^. This 

occurs with a probability P(I.(S)/ J.(S)) for a particular 

I.{S) having this property. Hence the expected rate 

of correct identification for the noisy sample 

J.{S) is: 

3 

2]P(Ii(S)/J3(S)). 
G(Ii(S))=di^ 

This is equivalent to: 

2o(G(Ii(S)), dj^)P{I.(S)/J.(S)) 

i 
or: 

l/P{Jj(S))2]0(G(Ii{S)), dj^)P(I.(S),Jj(S)). 

i 
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For a given J.(S), the rate of correct recognition 

is obtained by maximizing this sum. This 

optimization for a particular J-(S) is independent in 

its effect on optimizing the recognition of other 

Jj^{S). Hence this noisy-sample -wise optimization 

procedure provides an optimum recognition procedure 

for the set of all J.(S). 

J 

Theorem: The scene recognition procedure which minimizes 

the rate of false positive errors plus true negative 

errors is given by the function H(J-(S))=dj^ with the 

property that: 

2^M(G(Ii(S)), dj^)P(I.(S), Jj{S)) 
i 
is minimized. 

Proof: Similar to that of the preceding theorem. 

The last case is a bit more complex. 

Theorem: Suppose it is desired to keep the false positive error 

rate at or below a constant value a, while minimizing 

the true negative error rate. Then there exists a 

£, depending only on a, with the property that the 

desired optimality may be achieved by assigning to 

J.(S) the description d, such that 
J k 
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c.a(J.(S), dj^) + (l-c)b(J.(S),dj^) 
is minimal for all d^. 
Proof: 1) There must exist a description function having 

false positive error rate at or below a, and minimal 
true negative error rate. This follows from the 
fact that, e. g, , H(J.{S)) = gives a false positive 
error rate of 0. Hence the set of decision functions 
with false positive error rates below a is non-empty. 
There must exist one with minimal true negative 
error rate. Let it be denoted by H(J-(S)), and its 
true negative error rate by /8. 
2) Consider a decision function H*(J^(S)) which 
assigns to J(S) the description d^^ such that 
x.a(Jj(S)), dj^) + (1-x) b(J.(S), dj^ ) is minimal for all 
d^. We shall show that for a suitable value c of x, 
the function H*(J-(S)) has a false positive error rate 
at or below a, and a true negative error rate of /3. 
From this it will follow that the function H*(J^(S)), 
which is the function referred to in the theorem, has 
the desired optimality. 

27 



3) Let: 

a{j)=a(Jj(S).H(Jj(S))), 
b(j)=b(J.(S),H(Jj(S))); and also 
a*{3)=a(J(S),H*{J.(S))), 

-^ J X J 

b*{j)=b{J.(S),H|(Jj(S))). 

From the definition of H*(J-:(S))j we know that for 

X J 

all j, 

x.a(j) + (l-x)b(j) > x.a*{j) + (l-x)b*(j). 

Multiplying by P(J-{S)) and summing on j, we have: 

x^a{j)P{J-(S)) + {l-x)2^b(j)P(J.(S))> 

J J 

x2;a*(j)P(J.(S)) + (l-x)2]b*(j)P{J.(S)). 
j j 

But, by definition, 

2]a(j)P(J.(S)) = «, and 
J "' 

Xb(j)P(J.{S)) = fi. 

D ^ 

Also, letting 
ya*{j)P(J.(S)) = a*, and 

yb*(j)p(j.(s)) = /3*. 

^-> X J '^x 

j 

we have: 



xa + (l-x)y9 >xa* +fl*. 
X ^x 
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4) It is not difficult to see that x may be chosen so 

as to make «* = «. This follows by assuming that 

a* is a continuous function of x; and noting that 

oc-r = 0, since H* minimizes false positive errors 

ignoring true negative errors, and H(J-(S)) = 

achieves a false positive error rate of 0. By similar 

reasoning, a* increases without bound as x decreases 

from 1. Thus for some value £ of x the value of 

a* =a. This approach may be applied to the 

discrete case, where at least o^' will be very close 

to a. The expression of the previous step will then 

reduce to: 

ca + (l-c)fl > ca + (l-c)fl*. 

c 

But /8 is minimal, and (1-c) > 0, thus: 
^=/3*. QED 
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CHAPTER 2 
THE OPTLViAL USE OP REGIONAL INTENSITY INFORMATION 
IN THE DJCTECTION OF LINES 

I. THE REGIONAL QUALITY OF ISOLATED LINES 

In thiy section wc shall direct our attention to situations 
where there exist no systematic spatial relationships among the 
lines appc^aring in scenes under consideration. We shall briefly 
discuss the extent to which Hues in this case are "regional" in the 
sense that all intensity information relevant to the detection of a 
line is contained on the visual field withui a certain short distance 
from the line. 

Definition: Let S be a sampling region consisting of points 
uniformly distributed over a field of view. Let 
P(A, ii, S) be a predicate on intensity values at points 
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of S, which attempts to decide whether or not there 
is a line segment through points A and B. Let a 
and /3 respectively be the false positive and true 
negative error rates of the function F(A, B, S). 
Then the quantity 5t^(€) is defined as the smallest 
distance such that there exists a predicate F*(A, B) 
which decides on the existence of a line through A 
and B only on the basis of intensities of S lying 
within this distance of AB, and which has false 
positive and true negative error rates respectively 
below a+€ , and /3+€ . 
For some small value of €, the value of S (c) may take on a 
wide variety of values in different sorts of situations. For example 
8-p,(e) would be large in the situation where scenes consist of 
rectangular parallelapipeds. In this case, lines parallel to a 
proposed line, or incident with an end, constitute useful evidence 
for the existence of the proposed line. The spatial remoteness of 
useful evidence in this case might make 8Tr'(*) ^o* appreciably 
smaller than the radius of the field of view for small values of € . 
However, ^^.(t) may be relatively small in the case of scenes 
consisting of single discontinuities of intensity across the field, 
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or of sets of straight lines drawn at random across a sheet of paper. 
A hypothetical situation of particular interest is that in which scenes 
consist of lines which, within a relatively large radius, behave like 
lines given rise to by edges of plane -faced solids; but which are 
randomly distributed across the field of view. In cases such as 
this, two effects limit the predictive value of intensity information 
remote from a proposed line. 

The first effect has to do with the unpredictibility of 
intensity values away from an edge. In the absence of optical 
distortion and noise, intensities on a line normal to an edge might 
appear as in Figure 2. 1. The determination of the existence of an 
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INTENSITY PROFILE AT 
AN EDGE (Xq) 
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Figure 2. 1 
intensity discontinuity at x is dependent upon. finding the right and 
left limits of the intensity function at the point x„. The intensities 
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along the normal away from x„ contain information about the left 
and right limits at Xq. However the amount of information contained 
in these intensities falls off as one goes farther from the edge. This 
is due to possible unpredictible non -uniformities in the illumination 
and reflectivity of the objects in the scene. 

The second effect has to do with the nature of the blurring 
inherent in an optical system. An edge subject to such blurring 
appears as in Figure 2. 2. An abrupt step function is smeared out 

INTENSITY PROFILE AT AN 
EDGE (xq) subject TO 
BLURRING 

INTENSITY 



X, 







X 



Figure 2. 2 
a small distance from the edge and causes extreme values of the 
various derivatives of the intensity function in a small neighborhood 
of the edge. It is the extreme values of the derivatives that 
characterize and identify an edge. However^ the effect of the 
existence of an edge on the derivatives falls off drastically as one 
goes away from the edge. Hence, derivative information on a set 
of intensities remote from a possible edge can give little 
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information about the presence or absence of the edge. 

The spatial extent of this effect is determined by the 
critical resolution radius defined as follows. 
Definition: The critical resolution radius is the radius of a 
circle whose intensity equals (I+J)/2 around the 
image of a point of intensity I on a background of 
intensity J. 
This radius is related to such factors as the resolving power of 
the optics of the system. Empirical evidence suggests that for 
relatively small values of €, the value of S^i^) is on the order of 
magnitude of a few times this radius, in the hypothetical case 
mentioned of randomly distributed parallelopiped-edge-like lines. 

In situations where the existence of a line is indeed 
virtually determinable entirely from intensity information contained 
within a few critical radii of the proposed line, we can test for the 
presence of a line in a given location by examining an area which 
could only possibly contain a few other lines resolvable from the 
given line. In general there would seldom be any line but the 
central line in an examined region. This suggests the development 
of a theory of the optimal use of intensity information within a 
narrow rectangular region, based on a model which assumes the 
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possible existence of at most one line in the region. 

II. A REGIONAL VERSION OF THE GENERAL THEORY 

A regional version of the theory presented in the previous 
chapter will now be discussed. The theorems presented give rise 
to a procedure which is optimal in deciding on the presence or 
absence of lines within a narrow rectangular region. 

For the theory to be discussed, we adapt the following 
from chapter 1 : 

Let I I.(x, y) I be a set of real valued functions defined 
over a narrow rectangle R. 

Let D = I O, 1 [ be a two element description space. 

Let S, a sampling set, be a finite subset of R. 

Let F(I.{R)) be an identification function mapping onto D, 
with the inverse image of the element j^consisting of exactly the 
intensity functions given rise to by lines down the center of R. 

Let G(Ij^(S)) be a noise free sample identification function 
such that it and the function F{I.{R)) conform to the descriptive 
completeness criterion. In this case, descriptive completeness 
requires that any line identified on R by F(I.(R)) be identifiable 
by G(Ij^(S)) from intensities on S. 
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We shall make the following assumptions as well: 

1) That the intensity gathering device is subject to 
fundamental distortion, as defined in the previous chapter. 
In particular, we assume that this distortion governs the 
relationship between noisy samples J.(S), and corresponding 
noise free samples I.(S). 

2) That at most one line can occur in the set S. 

Some of these definitions and assumptions will not be explicitly 
used until the next chapter. In particular, the theorem about 
to be proved makes rather little explicit use of them. 

The optimization of the system in question will, as before, 
be a matter of making optimal use of a series of a priori and 
conditional probabilities in the minimization of certain error rates. 
In particular, we will assume that the a priori probabilities 
P(Ij^(S) of each noise -free sample I^(S), together with the 
conditional probabilities P(J-(S)/l.(S)) are known; or alternatively 
that the joint probabilities P(J.(S), I.(S)), the product of the previous 
two, are known. Again we may wish to hold the true negative 
error rate to some value and minimize the false positive error rate; 
hold the false positive error rate to a certain value and minimize 
the true negative error rate; or minimize the sum of the two. 
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A surprising result emerges from the theorem about to be discussed: 

that these three types of optimization are brought about by 

essentially the same procedure. 

We need the following definition before stating the main 

theorem of this section. 

Definition: 

Q(J(S)) =^P(I.(S)/J(S)) (2.1) 

G(Ii(S))=l 

This quantity is the probability, given the noisy sample J(S)j that 

there is a line down the center of the region R. 

Theorem: Suppose it is desired to keep the false positive error 

rate below a certain value a, and to minimize the true 

negative error rate. Then there exists a ^ such that 

the following decision function H(J(S)) satisfies the 

conditions: 

H(J(S))=l<=>Q(J(S))>/3 (2.2) 

Where (3 is the minimum value such that: 

J](l-Q(J.(S))P(J.(S))<a. (2.3) 

Q(Ji(S))>/3 

Proof: 1) Suppose a noisy pattern J(S) occurs, and that 

H(J(S))=1, i. e. a line is claimed to exist. Then 

(1-Q(J(S))) is the relative probability of a false 
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positive error whenever this pattern J(S) occurs, and 
(1-Q(J{S))P(J(S)) is the absolute probability of a false 
positive error involving the pattern J(S)). Thus: 

X](i-Q(Ji(s))P(Ji(s)) 

H(Ji(S))=l 

is the over-all false positive error rate of the function 

H(J(S)). 

2) It is clear that the value 1_ in place of /3 in 
equation (2. 3) satisfies it, and that such values are 
bounded below by 0. Hence there exists a least value 
satisfying the equation, which we shall call /3. 
Also, in the continuous case, for this value of the 
equality of equation (2. 3) will hold. One may 
consider this to be the case as well in the discrete 
case, if the number of discrete values is large. 

3) It remains to show that this procedure gives a 

minimum number of true negative errors. This error 

rate is given by: 

5^ Q(J.(S))P(J.(S)) (2.4) 

H(Ji(S))-0 

Suppose we consider another decision function 

H*(J(S)) satisfying the condition of having false positive 
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error rate equal to or less than a, i. e. 

2](l-Q(Ji(S))P(Ji(S))<o (2.5) 

H*(J.(S))=1 

We wish to show that its true negative error rate 

is not less than that of H(J(S)), given by expression 

(2. 4), i. e. that 

^Q(J.(S))P(J.(S)) > 5];0(J.(S))P(Ji(S)) (2.6) 

H*{Ji(S))=0 H(J.(S))-0 

4) Let R be the set of i's s. t. H(J^(S))=1, and 

H*{Ji(S))=0, 

and let T be the set of i's s. t. H(J^(S))=0, and 

H*(Ji(S))=l, 

and let U be the set of i's s. t. H(Ji(S))=0, and 

H*(J.(S))-0, 

and let V be the set of i's s. t. H(J.(S))=1, and 



H*(J.(S))=1. 



Then 



5];Q(J.(S))P(J.(S)) = 
H*(J.(S))=0 

J]Q(Ji(S))p(j.(s)) + XlQ<Ji(s))P(Ji(s)), 

U R 
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and 

J]Q(Ji(S))P(Ji(S)) = 
H(J.(S))=0 

XlQ(Ji(S))P(Ji(S)) + 2]Q(Ji{S))P{Ji(S)). 
U T 

So to prove (2. 6), we need to show: 

Jj3(J.(S))P{J.(S)) > 5]Q(Ji(S))P(Ji(S)) (2. 7) 

R T 

If i(R, thenQ{J.(S)) > /3,and if ifT, Q{Ji(S)) < /3, so 

2]Q(Ji(S))P(Ji(S)) > /S^ P(Ji(S)) (2. 8) 

R R 

and 

XQ(Ji(S))P{J.(S)) < /SX P(Ji(S)). (2. 9) 

T T 

Assuming equality in expression (2. 3), we have 

from (2. 3), (2. 5) and the definition of R, T, and V: 

2](i-Q(Ji(S))P(Ji(s)) = a > 2](i-Q(Ji(S))P(J-(S)), 

RUV TUV 

or J expanding and eliminating the sum on V: 

2P(Ji{S)) - ^Q{JAS)mj^{S))> 
R R 

2]P(Ji(S)) - 2]Q(Ji(S))P(J-(S)). (2.10) 

T T 
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Combining (2. 8), (2. 9) and (2. 10): 

i//3 2Q(Ji(s))P(Ji(s)) - 2]Q(Ji(s))P{Ji(s))> 

R R 

2]P(Ji(S)) - 5^Q(Ji{S))P{Ji(S)) > 
R R 

Xp(Ji(s)) - XQ(Ji(s))P(Ji(s))> 



l//32Q(Ji(S))P(Ji{S)) - 2;Q(J.(S))P(Ji{S)), 

T T 

or: 

{l//3-l)2]Q(Ji(S))P(Ji(S)) > (l//3-l)2]Q(Ji(S))P(Ji(S)). 
R T 

Formula (2. 7) follows from the fact that (1//3-1) is 

positive. QED 

The case of minimizing the false positive error rate with 

a fixed true negative error rate is almost identical: 

Theorem: Suppose it is desired to keep the true negative error 

rate below a certain value a, and to minimize the 

false positive error rate. Then there exists a^ 

such that the following decision function satisfies 

the conditions: 

H{J{S))=1<=> Q(J{S))> /3 . 

Proof: As for the previous theorem. 

It follows from these two theorems that whatever the value 
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of /3 used, the procedure of thresholding Q(J-(S)) is better than 
all other' procedures with the same false positive error rate, 
and better than all other procedui-es with the same true negative 
error rate. In this respect it is similai' to a saddle point solution 
of classical 2 by 2 game theory. 

It is not difficult to see that this procedure can, for a 
suitable choice of /3 , be used to minimize the sum of the two 
error rates. Consider an ai-bitrary choice procedure II* which 
gives a false positive ei-ror rate a and true negative error 
rate b. Then there exists a /3 such tliat thresholding Q(J(S)) at 
R gives false positive error rate a and ti'ue negative error rate 
b-', whei'e b* < b. This wTll provide a decision procedure 
whose total error rate is a r b* < a - b. Hence thi-esholding 
of C(J(S)) witl:i tills /3 gives a minimal sum of error rates. 



III. THE KHLATIONSHIP BETWEEN GEOBAL AND 

MEGIONAL EINE PREDICATES 



This section discusses the relation between the global 
line finding pr-oc(->dui'e described in chapter one, and the 
regional procedure of tlie present chapter. It is clear that the 
procedure of chapter one is computationally unfeasible. 
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Howevei' a y ysleniatic-, application of the r-egioiuil predicate to a 
large numbei- of regions on the field of view may be computationally 
practical. The latten' procedune, in certain cases of interest, can 
not acliieve the optimality of the global procedur'e. However it 
does trade a cer'tain amount of accuracy for computational 
feasibility. 

It has been pointed out that in certain cases whei'e lines 
on the visual field bear no systematic relationship to each other, 
a regional line predicate of the soft descx'ibed in the definition 
at the beginning of this cliapter seems to be pottmtially as 
accurate as a global one. It is tempting to formalize this notion 
into sotne soi't of theoi'cm, stating conditions under which a 
systematic application of a regional j3redicate of this sort to a 
suitably large numbcn- of regions on the field is equivalent to a 
global procedure. Lnfoi'tunately, an exact correspondence 
between these two pr'ocedurc^s is difficult to make. One problem 
is illustrated in Figui-e 2. 3. It is reasonable to assume that in 
the system cic^sci-ibed in cliapter one, if an element of the 
description space contains a segment AB, it would not also 
contain a subsegtnent of that line. More informally stated, the 
scene descriptions would not contain redundant segments. 
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A REGIONAL LINE PREDICATE 
APPLIED TO MANY REGIONS ON 
THE FIELD WOULD REPORT LINES 
AB, AC, . . . , AZ, BC, . . . , BZ, ETC. 
A GLOBAL PROCEDURE WOULD 
OMIT THE SUB-SEGMENTS AND 
REPORT ONLY AZ. 



Figure 2. 3 
On the other hand, the systematic application of a regional 
predicate of the sort mentioned would not only give positive 
identification for some complete line in the scene, but for all 
examined subsegments as well. Another difficulty is that the 
regional predicate might report the existence of two nearly 
coincident lines at very slightly different inclinations, where 
there is actually only one line. These difficulties will be 
encountered in chapter four, which describes an actual computer 
program embodying a regional line predicate. In the program, 
the problems are handled at the heuristic level. However, from 
a formal point of view, obvious efforts to solve them appear 
artificial. We will thus confine the remainder of this discussion 
to the informal level. 
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It is convenient to informally divide the evidence relating 
to the existence of a particular line into three categories: 

1) Global evidence, namely evidence from areas spatially 
remote from a possible line; 

2) Regional evidence, namely evidence from an area 
within a small radius of a possible line; 

3) Local evidence, namely evidence contained within a 
neighborhood of some point on a possible line. 

It has been pointed out that there are certain problems 
associated with a formal identification of a systematic use of a 
regional predicate with a global predicate. This identification 
appeared plausible only in cases where there is no relationship 
among the lines in the figures. However, the situation of 
particular interest is that in which the scenes consist of sets of 
plane-faced solids. In this case there is definitely a relationship 
among the lines in the figures; and a regional predicate cannot 
possibly achieve the optimality of a global one. Figures 2. 4 
and 2. 5 illustrate the possible failures of a regional line 
predicate, which might be avoided by the use of global evidence 
as well as regional evidence. 
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plausibility to closed line figures. Figure 2. 5 illustrates a 




^A 



THE REGIONAL PREDICATE 
APPLIED TO "A" MIGHT CLAIM 
THAT THERE IS NO LINE. 
HOWEVER, GLOBAL EVIDENCE 
STRONGLY SUGGESTS THERE 
IS. 



Figure 2. 5 
situation similar to that in Figure 2.4. In this case a line is 
totally ignored by the regional predicate, despite considerable 
global evidence for its existence. 

In chapter four, a program will be described which uses 
a regional predicate in the analysis of scenes consisting of 
prismatic solids. It is convenient to here describe briefly two 
features of this program, as the current context justifies their 
use. Specifically, the two features are designed to overcome 
the non-globality of the regional predicate as illustrated by 
the previous two examples. The sort of problem illustrated in 
Figure 2. 4 is overcome by the use of what will be termed links, 
which tentatively extend a line to a possible vertex. The sort of 
problem illustrated in Figure 2. 5 is handled by the procedure of 
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proposing lines. The procedure in effect assigns a high a priori 
probability to lines suggested by lines already identified. This 
tends to augment regional evidence for the proposed line. 

A final fact should be pointed out in this context concerning 
the relationship between the use of purely local evidence and the 
use of a regional predicate. This is analogous to the relationship 
between the systematic use of a regional line finder and the use 
of a global one. In the latter case, it is necessary that the lines 
in the scenes be spatially independent in order that the two 
procedures be equally effective. In order for purely local decisions 
as to the existence of a line to be as effective as regional decisions, 
it would, by analogy, be necessary that the points along a particular 
line be spatially independent. This is, of course, totally contrary 
to the nature of a line. It is thus quite impossible that a 
systematic application of a simple local decision procedure to a 
suitably large number of neighborhoods covering a region could 
be as sensitive in detecting lines as a regional predicate. By a 
"local decision procedure" is meant a procedure which decides 
entirely on the basis of intensity information from within a 
neighborhood of a point, whether or not a line traverses the 
neighborhood. The output of such a procedure would be a set of 
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points on the visual field, which would be incorporated into lines 
by finding maximal sets of adjacent colinear points. 

This two stage procedure is a special case of a set of 
procedures which process intensities in a neighborhood, and then 
process the results for a set of neighborhoods contained within a 
region. Although the particular two stage procedure mentioned 
in the previous paragraph can never be as sensitive as an optimal 
regional line finder, there exist two stage procedures of this sort 
which are. This is not surprising, since all regional predicates 
fall into this class, if one regards the first stage to be the 
identity function, and the second stage to be the regional predicate 
itself. There also exist non-trivial examples of members of this 
class which are as sensitive as an optimal regional procedure. 
The procedure described in the next two chapters involves such 
a two stage predicate. For the first stage, four values are 
obtained from the intensities in a neighborhood. Sets of these 
four-tuples, obtained from all the neighborhoods within a region, 
are processed by the second stage; and a decision is made as to 
the existence of a line in the region. The procedure may be 
regarded intuitively as extracting all necessary local evidence in 
the first stage, and employing all necessary regional evidence 
in the second stage. 
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CHAPTER 3 
COMPUTATIONAL APPROACHES TO THE REGIONAL THEORY 

I. INTROI3UCTION 

The object of this chapter is to give a computationally 
feasible niethod for computing values of the function given by 
Formula 2. 1 of the last chapter: 

Q(j(s)) - Y.nus)ij{s)). 

G{l^{S))=l 

It will be recalled that this is a function of the finite set J(S) 
of intensitit}S obtained by a particular sort of optical input 
device from within some region on the field of view. The 
thresholding of this value provides an optirnal regional line 
d (n e c t i o n p r o c e du r e . 

Prom Bayes' rule it follows that 

P(,){S),Ii(S)) 
P(I,(.)/J(S)) = ^^j,^,^ 
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so that 



Z 



P{J(S), I.(S)) 
Q(J(S)) 



G{Ii(S))=l 



P(J(S)) 
Using the relation 

P(J(S)) = J^P{J{S),l^{S))+ Y.P{J(S),liiS)). 
G(I-{S))=1 G(Ii(S))-0 

and 

P(J(S), I.(S)) = P(J(S)/I.(S))P(I.(S)), 
we have: 

Q(J(S)) = 



XP(J(S))/li(S))P(I.(S)) 



G(I.(S))-1 



2]P(J(S)/I-(S))P(I^{S)) ^ 2]P(J(S), I.(S))P(I.(S)) 
G(I-(S))=1 G(Ii(S))-0 

Sections II and III of this chapter will be concerned with 

the calculation of the values of P(J(S)/l^(S)). Sections IV and V 

propose a model for the relative probabilities of the various 

noise-free patterns Ij^(S). On the basis of the values of 

P(I^(S)) given by the model, it is possible to express the sums 

in the above expression as integrals, and to give a relatively 

simple explicit formula for G}(J{S)). In the last section of this 

chapter, this formula is applied to some simple cases of 

image-dissector output. 
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11. NOISE -FREE PATTERNS AND BLURRING DISTORTION 

The value of the conditional probability FiJjl.) is a 
function of the relation between the two sets of intensities 
J- and I., This relationship can be decomposed into: 

1) a determinate relation between the noise -free 
sample L and a distorted but noise -free version I*; and 

2) a simple probabilistic relation between this I* and 
the noisy pattern J-. 

The former will be discussed in this section; the latter in the 
next section. 

According to the definition of fundamental distortion^ 
the relation between a noise-free pattern I. and a distorted 
version I* may be expressed in terms of a convolution of the 
former with some function f(x, y). Possible examples of 
f(x, y) are given in Figure 3. 1. Theoretically, from the way an 





HILL FUNCTION PILLBOX FUNCTION 

Figure 3. 1 
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image -dissector is designed, this function should be cylindrically 
shaped as in the right hand illustration of Figure 3. 1. In fact, 
it is more like a gaussian "hill" as shown in the left hand 
illustration. An intensity function I(x, y) is acted upon by this 
convolution function f{x, y) to produce a blurred intensity 
function I*{x, y) as follows: 



00 00 



<{x, y) = / / f{u, v)I(x-u, y-v)du dv. 



-00-00 



If we had an edge which is reasonably uniform for a 
relatively long distance compared with the "width" of the 
convolution function, we can describe the convolution effect, at 
least locally, by means of a single integral instead of a double 
integral. In particular, suppose for some local region the 
intensity function I(x, y) represents a line parallel to the y-axis. 
Then locally we may write I{x, y) = I(x) and the corresponding 
blurred but noise-free intensity function I*(x, y) may be 
expressed in this region as: 



00 00 

I*(x,y) = / /f(u, v)I{x-u)du dv 



00 oo 



-00-00 



00 



/kx-u) /f(u. 



-00 



00 

v)dv 

-00 
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du. 



Since the inner integral is a function only of u, we may 
indicate it with a single function name: 



00 

g( 

-00 



00 

u) = / f(u, v)dv. 



and we have at a reasonably extensive and uniform edge 
parallel to the y-axis: 



00 



I*(x,y) = I*(x) = /^I(x-u)g(u)du. (3.1) 



-00 



We may obtain g(u) empirically by scanning at right angles 
across a very narrow line in the visual field. Let the intensity 
along a line normal to a vertical line in the visual field be 
given by: 

I(x) ==b if x<-a, x>a 

I(x) = c -a < X < a. 

Then the distorted version of the intensity locally around the 
line is given by: 

00 -x+a 

I*(x) = / I(x-u)g(u)du = c/g(u)du + b. 



-00 



-x-a 
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But Lim 
a-$>0 



- /g(u 



x+a -\ 
)du 



L -x-a 



= c.a-g(-x), so 



I*{x) = c-a-g(-x) + d, 
for very narrow lines. 

Figure 3. 2 shows plots of intensity taken along normals 
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Figure 3. 2 
to various vertical lines. According to the preceding 
paragraph, these curves should be of the form a'g(x) + b. 
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They are shown superimposed and normalized to have similar 
amplitudes. The lines are all quite narrow relative to the 
width of the hill plotted, and represent lines of a wide variety 
of intensities. The values are all proportional to absolute 
intensity, having been suitably transformed from vidissector 
output which is logarithmic. The degree of variation of 
un-normalized amplitude was greater than ten to one; and may 
be observed from the relative noisiness of the curves. All 
were recorded at a fixed noise level, and the amount of local 
fluctuation of a given line in the illustration is a measure of 
the degree to which the hill in question was enlarged in the 
normalization process. It should be noted that the shape of 
the curves is remarkably consistent. This indicates a good 
agreement between the theory just discussed and the actual 
behavior of the blurring process in the vidissector. Figure 3. 3 
illustrates a curve obtained from the data in Figure 3. 2 by 
hand fitting. Again, this curve should be of the form 
a-'g(x) + b, where g(x) is the one -dimensional blurring function. 
Also illustrated is a curve of the integral of this function, 
obtained by hand integration. This latter curve will be 
compared with a similar curve obtained by two different methods 
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HAND FITTED CURVE 
OBTAINED FROM 
CURVES IN FIGURE 3. 2 
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INTEGRAL FROM TO X 
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OBTAINED BY HAND 
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Figure 3. 3 
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which will now be described. 

Another approach to finding g(x) is based on the fact 
that the peak value of an intensity scan normal to a line of 
non-negligible width a is related in a simple way to the integral 

a/ 2 
'g(x) dx. 



f' 



-a/2 
namely, if the actual intensity along a normal to a narrow 
vertical line is given by: 

I(x) = c ifx<b, x>b + a 

I(x) = c+d b<x<b+a, 
then I*(x), the distorted intensity function is given by: 

00 00 

I*(x) = / I(x-u)g(u)du = I {c + I^(x-u))g(u)du, 

-00 -00 



where I (x) = except in the interval (b, b+a), where 
I (x) = d. One may write the latter integral as: 



00 00 

:/g(u)du + / I^(x-u)g(u)du, 



But the former integral is a constant, and the latter integral 
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1 /g(u)du. 



is equal to: 

b+a-x 

d 

b-x 

and has an extremum at x == l/2(a + b), which is a minimum or 
a maximum depending on the sign of d. At this extremum, the 
integral has the value: 

a/2 



d /g(u)du. 



■a/2 

Thus a scan made perpendicular to a line of width a and 
intensity c+d on a background of intensity £ has a maximum 
(or minimum) whose value relative to the intensity somewhat 
away from the line is: 
a/2 



d /g(u)du. 



-a/2 
Suppose a scan is taken across a pair of narrow 
triangles as in the left of Figure 3.4. In the figure, 
kLn(l/ intensity) is plotted for successive scans. In the right 
hand side of the figure, the values of the maxima of each 
scan minus the background value are plotted for each scan as 
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Figure 3. 4 
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a function of the spatial location of the scan along the set of 
triangles. Since the width of the triangle at a particular scan 
is a linear function of the position of the scan along the 
triangle, the plot just mentioned may be considered to be a 
function of the width of the line being scanned. Hence the 
X-axis in the plot is in units commensurate with this distance 
measure, and in the same scale as is used in Figures 3, 2 
and 3. 3 relative to the physical dimensions of the vidissector. 
The values plotted are thus those of kLn(l/m(x)) - kLn(l/c), 
where m(x) is the actual minimum (since the target is dark) 
of intensity along a scan of a line of width x, and c is the 
background intensity. If the blurring function behaves 
according to the theory proposed, we should have: 

x/2 x/2 

c - m(x) = d/g(u)du = 2d /g(u)du (3.2) 

-x/2 

where d is the absolute value of the intensity difference between 

the background and the target. Using the curve of the integral 

of the function in Figure 3. 3, and doing a certain amount of 

arithmetic involving logarithms, we can obtain a curve of 

predicted maxima in accordance with the premise expressed 
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in (3. 2), and with the assertion that the curves in Figure 3. 3 
represent g(x) and its integral. This curve is indicated in 
Figure 3. 4 as the dashed line superimposed on the plotted 
curve in the figure at the right. The closeness of these two 
curves indicates good consistency between these two approaches 
to finding g(x); and provides further evidence that the curves in 
Figure 3. 3 represent g(x) and its integral. 

A third approach to the computation of g(x) is based on 
the fact that at a wide edge, the value of the intensity along a 
line normal to the edge is related in a simple way to the 
function: 



y 
h(y) = y g(x)dx. 

— 00 






In particular, if the intensity function for a verticail edge is 
given by: 

I(x) = a x < 

I(x) = b X > 0, 
then the distorted value of intensity, I*(x), is given by: 

00 00 

I*(x) = / I(x-u)g(u)du = a + b /g(u)du. 
-°° -X 

62 



This equality is derived by an argument similar to that given 
above. In Figure 3. 5 we have the values of kl^n(l / intensity) 
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^n ^ 7... SCAN 

LINE 
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Figure 3. 5 
for the intensities along a line normal to an edge, subject to 
the blurring of the vidissector. The illustrated points along 
the curve repi-esent values obtained from the integral curve 
in Figure 3. 3, suitably scaled. For clarity, the curve 
represented by these points has been omitted. The close 
correspondence between the points and the curve further 
verifies that the curves of Figure 3. 3 are a good approximation 
to g(x) and its integral. 



III. NOISE AND THE COMPUTATION OF P(J-/l.) 



A given noise -free sample I- turns into a distorted 
sample \r as shown in the previous section by the convolution 
of I. with son:ie blurring function. The conditional probability 
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P(J-/L) may easily be expressed as the probability that simple 
noise on the sample I* would give rise to the configuration of 
intensities J.. 

It has been claimed that a large number of intensities 
read at a given point under a given lighting situation by the 
vidissector have approximately a normal distribution. 
According to the way in which the vidissector is constructed, 
the absolute noise level is a constant for the logarithmically 
scaled intensities delivered. Actually there are four possible 
noise levels; however all work in the remainder of this paper 
will involve only one, the most accurate. Hence we shall 
assume that there is a single constant noise level. The 
intensities read at a particular point for a particular set of 
lighting conditions follow the distribution: 



1^2^ ^ 



2LaJ 



dx. 



aV2) 

where a is a constant. From this may be computed the 
probability of a given configuration of intensities 
V , V , . . . , V at points with actual intensities 
u^, u , . . . , u . Since the noise-induced deviations are 
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statistically independent, this probability is: 

m 1 fUj -Vj 1 ' 

n l /J 21 a J 
172^ r dvidv2. . . dv^, 

i=l 
or, letting 

B..=i:(u,-./ (3.3) 



with 



I* = (u , u , . . . , u ) 
i 12 m 



J. = (v^, Vg, ... ,v^). 



we have: 



1 /, 2 Lad 



P(J./li) = P(J./lr) = j^„ e "■ •" dvidv2. . . dv^. (3. 4) 

The value of a for the signal to noise ratio has been reported 
(Horn 1969) in terms of the percentage error of a logarithmic 
intensity value corresponding to an error of one standard 
deviation of the errors at that intensity. Again, this 
percentage is a constant for all intensities, and is reported to 
have the value 1. 6%. Since the logarithmic intensities are 
given by: 

lOOgLog^d/l), 
the absolute error corresponding to one standard deviation 
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is: 

a = 64. Log, (I- .0161) - 64. Log(I) =1.5 (3. 5) 

2 <^ 

This is the value of a which will be used in (3. 4). 

The procedure for computing the conditional probability 
P(J(S)/I.(S)) is to determine the blurred version Ir(S) of the 
set of intensities I^(S), and to calculate the value of 
P(J.(S)/lr{S)) by (3. 4). The relationship between I.(S) and 
p;=(S) is in general governed by the function f(u, v) referred to 
in the previous section. However we shall consider only 
cases where (3. 1) applies, and will need only the function 
g(x) to determine this relationship. 



IV. A MODEL FOR THE SET \It\ AND COMPUTATIONAL 

CONSEQUENCES 



In this section we shall give models for the sets 
]lr(S)[ of noise-free distorted intensity configurations teiken 
over particular rectangular regions S. Our attention will be 
confined to sampling regions S which have a particular width, 
and whose lengths are integral multiples n of the width. We 
may consider such a region to be composed of n square 
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sub -components, denote it by S , and use the definition: 

I*(S ) = I*", 
in 1 

Rules will be given for assigning a priori probability values 
P(I*. ) to the elements of jl*. } . It will be recalled that there 
is a determinate 1:1 correspondence between elements I*, 
and elements I.(S ). Thus the analysis in section I of this 
chapter remains valid with I*j^ in place of I.(S). It follows that 
the value of Q(J(S)) of (2. 1) is determined by (3. 4) and by the 
values of P(I*?). The latter are the subject of this section. 

Consider, for a fixed n, the set |l*. [ of all noise-free 
distorted samples over rectangular regions composed of n 
square sub -components. We shall divide this set into four 
subsets as follows: 

Let CL be the subset of |l*. [ consisting of sets of 

intensities from lines in the visual field exactly 

centered in the rectangular region. 

Let CE be the subset of 1 1*- | consisting of sets of 

intensities from edges in the visual field exactly 

centered in the rectangular region. 

Let CH be the subset of |l*. | consisting of sets of 
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intensities from a homogeneous area in the rectangular 
region. 

Let CS be the subset of | I*^J consisting of sets of 

" i ' 

intensities from lines or edges which are not centered 
in the rectangular region. 

We shall first consider a model of CL, and base it on 
the following assumptions: 

1) That the absolute intensities of the samples of CL 
are of no consequence, so that in each square 

sub -component of a sample I*? the sum of the intensities 
may be assumed to be 0. 

2) That on any section across some j-th sub-component 
of a member 1-.'^^ of CL, the intensity function has the 
form: 

a.^^g(x) + bi^j 

where a- • will be termed the relative amplitude , and 
1' J 

b. is chosen to conform with assumption 1). 

1.3 

3) That the relative amplitude a- ■ of an intensity peak 
(or valley) is uniform within the j-th sub-component of 



"-' (-if-iT-^^l^ Ti.<: iiicl^ L. 
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I'fi?. A sample I*, may thus be described in terms of 



an n-tuple of relative amplitudes (a. , , ... , a. ), e. g. 

1, i ij ri 



as I*(a. .,..., a. ). 
ij 1' ' 1, n' 



,.n 



4) That each sample I*, was given rise to from some 
line, highlight, etc., in the real world with an 
"idealized" relative peak (or valley) ainplitude a., 
which is uniform along the length of the line. 

5) That this idealized relative amplitude is purturbed 
randomly along the length of the line so that there is a 
minor fluctuation of relative amplitude a. ■ among the 

various sub -components of a sample I*. , with a 

2 
normal distribution with variance a . 

n 

The first of these assumptions ignores a certain amount 
of information about a line which is useful in distinguishing it 
from a non-line. Broadly speaking, it prevents a distinction 
between a series of peaks in a line whose absolute intensity is 
constant or regularly varying; and a series of peaks in a line 
whose absolute intensities vary randomly or dis continuously. 
This is not undesirable in the case of a line lying in the real 
world partly in shadow. However, it would be desirable to 
eliminate as lines a series of "lined up" intensity peaks whose 
absolute intensity varied randomly, as this would likely be due 
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to noise. 

The seeond assumption is justified on the basis of the 
discussion in the first section, and by the fact that we are 
considering noise-free but distorted samples. 

The third assumption is justified on the basis that the 
"ridges" of intensity of samples in CL are rather uniform 
along their whole lengths, and would thus be quite uniform on 
square sub-components. This assumption is particularly valid 
if a square sub -component is only sampled along a line 
traversing it perpendicular to the orientation of the rectangle. 
This is approximately what is done in the program which will 
be discussed in the next chapter. 

The "idealized" constant relative amplitude of 
assumption 4) may be thought of as the relative amplitude 
which would be obtained in the absence of physical defects of 
the object being viewed or on the vidissector photocathode, and 
in the absence of spurious local reflections and anomalies of 
lighting. The purturbations of assumption 5) may be thought 
of as a result of these defects etc. , e. g. a nick on the edge of 
a cube, a slight shadow or a bad spot on the photocathode of 
the vidissector. The idea of an idealized constant value of 
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the relative amplitude along an edge is further justified by the 
fact that intensities used will be logarithmic. Any ridge of 
intensity^ for example a highlight, is a relative contrast 
phenomenon, and may vary in absolute intensity, but not in 
relative intensity. A logarithmic intensity function would tend 
to register amplitude of highlight the same regardless of the 
absolute intensity. The same is true when the argument is 
applied to intensity discontinuities at an edge. 

It follows from these assumptions that if we let F(a) 
be the a priori probability of an edge in the real world with 
"idealized" intensity a, then the probability of existence of 
a particular I*. = I*(a. ,, ... , a- ) given rise to by an edge 
with idealized relative amplitude a and parameterized by 
amplitudes a. . , ... , a. is given by: 
n 1 ra-a^n^ 



__ra-a^= 



F(a)| |-;;1if^C; ^" "" -da.^^ ... dai^n 



The probability that this I*, would occur from a source with 
any idealized amplitude is given by the sum over a of the above 
expression. This sum is simply the a priori probability P(I*. ) = 

P(I*(ai^ 1' • • • ' ^i, n^- 

This sum may then be evaluated provided the distribution 
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F(a) is known. Assuming it is normal with standard 
deviation p , we have: 



n 



where P(CL) is the a priori probability of existence of 
a member of CL. We then have: 
P(I*{a. ^, ... , a^^^)) = 

.irariry (a-a,) ^-| 

W^.l^^ dada.^^ ... da.^^(3.6) 

a 

This sum, including the differential da may be expressed as 

as integral, with the exponent slightly rewritten: 

« _irg^ na' -2aXai| + Sa,', I 

e '^''- ^' -"da. 



/■ 



00 



The exponent may be further rewritten as: 
2 L LPn aU ^^ al J 



Letting: 



A - ^ + ^ 



of, PA 

B = -^V 



and completeing the square, the exponent becomes, in 
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terms of A and B: 



-i rA(a^-2a(B/A) + (B/A)*^ - BVA + SaVo^l, 

The integral becomes: 

^ -A/2(a - B/Af - l/2{2a2 l^l- B^A) 



■00 



The second term in the exponent is independent of the 
integration variable a, and may be taken outside the 
integral sign. The remainimg definite integral is well 



known to have the value-^Tr/A. Thus the integral is 
equivalent to: 



ie 



Safi 



o'„ 



A 



The result of substituting in this expression the values 

of A and B, of substituting the resulting expression into 

(3. 6), and of doing some straightforward algebraic mainpulation 



is: 



P(I*(a. ., 



P(CL) gn 



%n) 



if £afi ^ npg^(aii-a ,n 



^2no"„ M+ripl 



da. ...da. (3.7) 
1. 1 1, n 
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An analysis of the CE category may be made in a nearly 
identical fashion. In this case, the function g(x) of assumption 2) 
is replaced with the function h(x), its integral. As was pointed 
out in Section II, this function gives the intensity along a normal 
to an edge, and is the obvious correspond to g(x), which gives 
the intensity along a normal to a line. A formula for the 
a priori probability of a member of class CL may be derived 
from cin analysis nearly identical to that given above. The 
resulting formula differs from (3. 7) only in that different values 
appear for the variances a„ and p„; and that P(CE) is used in 
place of P{CL). 

Again, an analysis of the CH category may be made along 
nearly identical lines. In this case, the function g(x) is replaced 
with l(x) = ax + b, the function which gives the intensity profile 
across a homogeneous region. Again the resulting formula for 
a priori probabilities of members of CH differs from (3. 7) only 
in the values of a„ and p„; and in the use of P(CH) in place of 
P(CL). 

At this point it is convenient to recapitulate some of the 
foregoing by rewriting the formula for Q(J(S)) appearing at the 
end of Section I in terms of the specific notation introduced in 
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later sections, and some additional new notation. First, it will 

be recalled that the notation I*(a. ... , a- ) was introduced 

to refer to noise-free distorted samples over a region consisting 

of n square sub-components, and containing a line centered in 

the region. A value a. refers to the relative amplitude of the 

intensity profile of the i-th sample, taken across the j-th 

sub -component. We may similarly adopt the notation 

I*(b- 1 , . . . , b. ) to refer to noise-free distorted samples 
1, 1' ' 1, n 

from the class CE, and I*(c- ,, ... , c- ) to refer to samples 

1, i i, n 

n 
from CH. We shall reserve the notation I*- to refer only to 

distorted noise-free samples froin the class CS. We may 

introduce similar notation for noisy samples by a definition 

similar to that used at the beginning of Section IV: 

j"-Ji(V- 

By this notation is meant the i-th possible noisy sample on a 
region consisting of n square sub-components. 

The sets CL, CE, CH and CS provide us with an 
alternative for expressing the function G(L(S)) used in the 
formula for Q(J(S)) in Section 1. If we make the following 
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definitions ; 



CUJ"^) 



XP(j"/l*(ai^ 1 \J)P(I*<\r ••• ' %n>^' (2-^) 



1 
CE(j]') 



yP(j''/l*(b. ., ... , b. ^))P{I*(b. ... , b. )), (3.9) 



1 
CH(jJ') = 



yP(j"/l*(c. ^ c. ^))P(I*{c. ...... c. )), (3.10) 

1 



CS(J^) = Yp(j'^/I*")P(I*"). (3. 11) 

Then it follows from the 1:1 correspondence of distorted 

noise-free elements, and non-distorted noise-free elements 

that: 

XP(j5'/li(S^))P(I.(S^)) - CL(j") + CE(J^''), 
G(I.(S^))=1 



and 



y P(J^/I.(S ))P(I.(S )) - CH(J^)+ cs(j"). 
^-- J 1 n ' 1 n J J 



G(I.(S^))=0 
Consequently we may rewrite the formula at the end of Section I 

^^* n, , n, 

CL(Jj ) + CE(J. ) 

Q(J^) - n n n n~ (3. 12) 

^^ j' CL(J.) + CE(J. ) + CH(J.) + CS(J.) 
3 J J 3 
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We may now turn to an analysis of the class CS. It turns 

out not to be necessary to make an explicit model as was done 

for the classes CE, CL and CH. To see this, recall 

Formula (3. 12) of the previous paragraph, and Formulas 

(3. 8) - (3. 11), which define its components. An element in 

these latter sums, e. g. , 

P(j''/I*(a , ... , a ^))P(I*(a a )) 

J 1, 1 1, n 1, i 1, n 



from the sum CL, may be thought of as a measure of 
similarity between J. and the distorted noise -free sample 
I*(aj 1' ■ • • ' ^i n^' weighted by the plausibility of the latter. 
Thus the value of CS, for example, is roughly proportional to 

the plausibility or frequency of occurrence of members of CS, 

n 
and to the similarity of J- to members of CS. 

J 

If J- occurs over a rectangle S on the visual field 
which is centered on the image of a line, the value of 
CS(J^ ) is quite small relative to the other members of the 
denominator of Q(J- ), owing to the relative infrequency of 
members of CS as compared with members of CH, and to the 
fact that such a J. is as similar to a pattern of intensities 
from a homogeneous region as it is to a pattern of intensities 
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n 
containing a skewed line. Since the value of CS(J. ) is thus 

sniallj it could be oixiitted in this case, or at least approximated 

with the sum of values of 

k, n, ^„k, n, 
CL (J. ) - CE (J. ) 
J J 

k 
over a set of regions S which are nearly coincident with Sj^, 

but are slightly skewed with respect to it. These values may be 

obtained "for free" for any region S by a line -finder which has 

to be able to find lines in all positions in the visual field; since 

k, n^ k n 

it would necessariiv compute CL (J. } and CE (J ) in the 

J J 

process of testing the other regions S for the presence of lines. 

If S is a region in the visual field which did not contain 

n ^ 

n 
a line, a noisy sample J. over S would, by a similar argument, 

have the propex'ty that CS(J. ) makes a rather small contribution 

n 
to the denominator of Q(J. ), and could be similarly approximated, 

or indeed omitted. 

It is only for the case that the region Sj^ contains the 

n 
linage of a skew line that the value of CS(J^ ) becoines 

significant. However, this effect occurs only for rectangles 

approximately coincident with a rectangle S centered on a line 

n n 

in the visual field. Let Q':^(J. ) be the same as Q(J. ) but with 



the value of CS(J. ) omitted from the denominator: 
3 

CL(j'^) + CE(j'^) 
Q*(j'') = I I (3.13) 

^ cL(j") + CE(/) + cmj"^) 

J J J 

It has been empirically checked that Q*(J. ) obtains a maximum 

value locally approximately on the rectangle S^ centered on the 

n 

line. This is quite plausible, since the value of Q*(J. ) should 

be large for rectajigles approximately coincident with S , and 
if it didn't have a maximum on S , it would either be offset 
parallel to S^^, which is unlikely because of the symmetry of 

the intensity pattern; or there would be several maxima, 

n 
which seems not to happen. Hence the omission of CS(J. ) 

from the denominator of Q(J-j ) has the effect of giving 

erroneously high values on regions nearly centered on a true 

line, but not erroneously high values on the maximum value 

region, namely a region centered on the line. However, these 

values are not so augmented as to create false maxima for 

regions not quite centered on the line. 

In summary, the set of noise-free distorted samples 

is partitioned into four classes CL, CH, CE and CS, Our 

threshold function Q(J^) is expressed, in (3. 12), in terms of 
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four sums CL{J^), CH(J^), CE(j") and CS(J^) over these four 
J J J J 

classes. Explicit models of CL, CH and CE allow a simple 

explicit expression of the a priori probabilities of members 

of these sets. These probabilities, together with the results 

of Sections II and III, determine the values of the sums 

CL(J^), CH(j") and CE{j"). It has been shown that CSCj'^) 
J J J J 

may be computed indirectly, or even omitted from the 
denominator of Q(J. ). The latter is permissible, resulting in 
Formula (3. 13), in the case where the regional predicate is 
applied systematically over the whole field of view. 



V. COMPUTATIONS OF CL(j"), CE(j") AND CH(J^) BY 

MEANS OF INTEGRALS '' "' •• 



In the previous section, it was pointed out that the 
threshold function Q^J- ) could be approximated with the 
function Q*(J. ) given by (3. 13). This function is determined 
entirely by the values of sums CL(j"), CE(J^) and CH{j"), as 

J J J 

defined by (3. 8), (3. 9) and (3. 10). In this section, we shall 
show how to express these sums as integrals, and to reduce 
them to relatively simple closed form. 

It might seem a good strategy to evaluate the 
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expression, for example CL{J. ) as given by (3. 8), by- 
substituting values given by (3. 4) and (3. 7) respectively for 
the two terms within the summation sign, expressing the 
resulting sum as an integral, and finding a closed form for 
the value of the integral. This approach appears to run into 
difficulties, and a slightly different approach will be used 
instead. 

Our first step is to express, e. g. , CL.(J. ) in a more 

J 

convenient form. Since the sum in (3. 8) is taken over all 
n-tuples, CL.(J^) may be expressed as: 



Z Z ••■• 

a a 
i, 1 i, 2 



a. 
1, n 



where the sum over a. . means that a^^ -^ in the summand 

takes on all positive integral values. We may express 

P(I*(a. ^ , ... , a-^ ^)) as 

2]P(I*(ai^l , ... , a.^^)/a)P{a), 
a 

where P(a) is the a priori probability of a line with 

idealized amplitude a; and P{I*(a. ^, ... , a. „)/a) is the 

relative probability of occurrence of a noise -free distorted 
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sample with sub region amplitudes a. , ^ . . . a^ , given 
a line with idealized amplitude a. The conditional probability 
P(I*(a- 1 .... J a- )/a) may be expressed^ as in Section IV, 
as a product of probabilities: 

n 
k=l 
where P(a. i, / a) is understood as the probability^ given a line 
with idealized amplitude a, that the k-th sub -component of 
a noise-free sample will have a relative amplitude a. , . The 
above probability may be expressed as a product, since the 



elements in the product are statistically independent 



Likewise, by statistical independence, P(J. /l*(a. -, , ... , a- )) 



may be expi'essed as a product: 

n 
P(j"/I*{a, .,..., a. )) == Tlp{j'Hk)/a. ,), 

ri 
where J ^ (k) is understood to mean the set of intensities of the 

noisy sample j" which lie within the k-th square sub -component 

of the region. Substituting and rearranging, we have: 

CUJp -~ 

xp(a)x •••z np('Vk)/^k)p(-i,k/^)- 

a a . , a k = 1 

1, 1 i, n 
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It is not very difficult to sec that the product over k and the 
summations over the a. , 's may be interchanged, so that: 

1, K 



cL(j;^) = 

a k=l a^^^ 

n n 

Similar expressions exist for CE(J. ) and CH{J. ). 

An expression for P(j"(k)/a. ) may be obtained by 

adapting (3. 4) to the present context, using the value of a 

from (3. 5): 



1 D, 



ik 



P(T{k)/a )= L dv. , . . dv, K, (3. to) 



where 



^ . ^ t .^ 



m 

D. , , =y (u; , - v' , r, (3.16) 

1. J.k ^^ lA j,k 



which is similar to (3. 3) in tliat it refers to some i-th noise- 
free distorted sample and a j -th noisy sample, but has a 
third subscript to indicate it involves only the k-th 
sub-component. The values u- , are the m intensity values 
within the k-tli sub -c:omponent of the i-th noise-free distoihed 
sample containing a line; and are taken over' points which iuay 
be denoted by pj_ ,^. Since the amplitude in this region is 
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a. , , we have by assumption 2) of Section IV that the intensities 

1, K 

at points p^ ^ are given by: 

"i, k = ^i, kg(4, k) + ^i, k' 
where Xj^ j^ is the perpendicular distance between p^ j^, and a 

line traversing the center of the region in the direction of the 

expected line. If the geometry of the points p- j^ is fixed for 

all samples and sub -regions, then the value of g{xj^ j^) is 

independent of the sample and sub region, and we may 

express such an intensity value by g{x ). By assumption 1) 

of Section IV, we may choose b. , so that the sum of u^ i^ is 

zero, i. e. , by choosing b. v, = g. Hence defining values 

(U^ ... , U"^)by: 

U^ - g(x^) - lg{x)lm, (3.17) 

t 

we may express the m intensities in the k-th sub -component 

of the i-th distorted noise-free sample by: 

,1 m . 

(^i,k' ••• ' ^i,k) = 

(ai,kU^ ... , ai^kU™) (3.18) 

where: 

E U* = 0. (3. 19) 

t 



84 



To be consistent with assumption 1), we should also assume 
that: 

^4^=0. (3.20) 

The value of D- , of (3. 15) may be expressed in terms of the 
ij 2' '^ 

values just defined as: 

It is easy to show that: 

Di,3,k = b(ai,k-a^,k)'-^\k' ^''^'^ 

where: 



a? 



j'k E(ut)^ 



iiii (3.23) 



t 
J] (uS^ (3.24) 



t 



a-^? ,uS^. (3.25) 



Combining we have^ omitting the differentials dv- ^: 

1 rb(ai, - a>:jj' + R^, 1 

We may express P(aj^ j^/a) as before by assuming a 
normal distribution on the various values of aj^ ^, with 
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standard deviation o and mean a: 

n ~ 



1 

P(a. /a) - — rr 

1. ^ V2 rr o^ 



a - a: 



a 



" J da 



i, k- 



(3.27) 



Combining (3. 14), (3. 26) and (3. 27) we have, again omitting 
t 



differentials dv 



. k' 



n 



n 



ci.(j,) = Xt^^^^n z 



1 



k=l aT , V^^n{ 1.5/2^)'^ 

1, k 

b(a,k - a=::, )' + R., {a - a;,)' 



(1.5) = 



-■ da., da (3. 28) 
ik 



witli b, a-^^ , and R. given by (3. 23), (3. 24) and (3. 2o). 
inner sum may be put in the form: 

9 

-Re ■ 

■^z e 



The 



-Ba. , + Ca. , - D 

^' k da 



i,k^ 



i> k 



wiiich admits of the appi'oximation used in the preceding 



section: 



-Bx'^ -r Cx + D 



2 
■ Bx ^ Cx - D 



dx 



91 

A\^ P 4B 



— D 



Vb 



Applying this approximation to the inner sum in (3. 28), we 



have after considerable algebra: 



CUJJ) :^ S P(a)n^ (y2¥l.5ry/(1.5)*+ba„^ 

' RjK , b(a'^, - a)' 1 
(1.5)' (1.5)^ + ba„2 J^^^ 



1^ 



where again, the diffentials dv have been omitted, 
may alternatively be expressed: 



CL(J") ~ YF''^^ 



1. 5 




U(V2^1.5)'"V(1.5)' + ba„' J 


SRj. bX(a'|v - ar 

JS + 5- • 





.(1.5f (1.5)2+ba„' J 



da. 



and expressing P(a) as before by: 



P(a)= ^^^Q 

this latter expression for CL(J") may be put in the 
form: 



n. 



AZe 



-Ba'^ + Ca+ D 



da. 



This 
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and may be approximated as before to yield: 



CL(j") ~ P(CL) 



1. 5 



(/2Fl.5r>/(l-5)'+ba^_ 



"Vb^ToTsf 



TTT, — 7^ 



^ba^+nbpn+d.Sy 



i rSRj. ^ b^a^^' ^ nbit^S(a>^- a^^ - . 

p'2[(1.5)^ (1.5)2+ba^nbp2„ (bo^+(l. 5)^)(b<^+nbp„'+(l. 5)'J 
*^ ,(3.29) 

t 
with differentials dv omitted. Similar expressions exist 

for CE(j") and CH(j"). 
1 5 



VI. APPLICATION OF THE THEORY TO SOME ACTUAL 

VIDISSECTOR OUTPUT 



In this Section we shall apply some computational 
procedures suggested by the foregoing theory to intensity 
values obtained from the vidissector. These intensity values 
will be taken along single lines normal to various lines and 
edges in the visual field. The results of the computation 
will serve to validate the theory presented, and justify 
some assumptions and approximations in its computational 
implementation which will be used by the program described 
in the next chapter. 

In applying the foregoing theory of a regional line 
predicate, it is first necessary to choose a suitable physical 
width for a rectangular region. It appears that, for the 
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vidissector in current use, this width should be between 25 
and 50 units, where one unit is 1/2000 of the full field width. 
This range of values is chosen to conform with the assertion 
that, in the terminology of chapter two, the majority of 
regional evidence for the existence of a line is contained 
within a neighborhood of the line whose radius is on the 
order of a few times the "critical resolution radius" of the 
vidissector. This latter quantity, as defined in Section I 
of chapter two, may be easily obtained by examining 
Figure 3. 3. It may be seen that the intensity in the upper 
curve falls to one half of its maximum value at a radius of 
three units from the center of the curve. This value is 
approximately the critical resolution radius as previously 
defined. For present purposes, we took "a few times" to 
mean specifically eight times, which yielded a region width 
of 50 units. Later results, reported in the next chapter, 
indicated that this value may have been conservatively large, 
and somewhat better results seem to have been obtained with 
a region width of 2 5 units. In any case, the value of 50 units 
turned out to be adequate for present purposes. 
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It seems appropriate to begin an empirical investigation 
of the properties of the regional line predicate here developed 
with a consideration of the case where the region to which it 
is applied consists of a single square sub -region. This 
admits of a simplification in the formula for Q*(Jj^) as given 
by (3. 13) and (3. 29). In particular, the term involving 
(a* , - a* ) in (3. 29) becomes zero, since a* , is the 
average of only one term. Thus this term is eliminated from 
the exponent in (3. 29); and likewise similar terms are 
eliminated from corresponding expressions for CE(j") and 
CH{Jj^). 

Another simplification of the exponent in the expression 

for CL(J. ) given by (3. 29), and likewise for corresponding 

expressions for CE(J. ) and CH(j"), follows from a 

J J 

consideration of the relative magnitudes of the values 

2 2 

(1.5) and ha- . The latter is the value of b given by (3. 24) 

times the variance of values of idealized relative amplitude 

of the various lines in scenes in the real world. Since the 

distribution of these values has a mean of zero, it follows 

from elementary properties of the normal curve that about 

1/3 of all lines have idealized relative amplitude a with the 
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property that |a| > cr • Since the variance of the sub-region 

amplitudes a- ■ is small relative to the variance of the values 

of a, the same holds true of these amplitudes, i. e. , about 1/3 

of all sub -components of noise-free samples containing lines 

have amplitudes a. . with I a. . I > o" . If the intensities 

within a sub -region of a noise -free distorted sample are given 

by (uj , , ... , u. j^) and their spread is defined as: 
1 = ^ 



Vi:<<k-\/' 



with 



then it is easy to see from (3. 18) that this spread has the 
value a. \jh. where b is as defined in (3. 24). It follows 
from what was said previously that about 1/3 of all sub- 
components of noise-free samples have a spread of greater 
than Vbo- . More informally stated: -^o" is a not uncommonly 
large value of the spread of intensities in sub -components of 
noise -free samples containing lines. In the case of noisy 
samples containing lines, empirical investigations suggest 
that a spread of 30 is not uncommon; and it is easy to see 
from the fact that noise level is relatively much smaller than 
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2 2 

bo-^ » (1. 5) . 



30, that this value is also not an uncommonly large value of 
spread for noise- free samples containing lines. Consequently: 

(3. 30) 

Now by simple algebra the exponent in (3. 29) may be written, 
excluding the third term which is zero, and using the relations 
in (3, 23), (3. 24) and (3. 25): 

~2 (1. 5)' ~ (1. 5f((l. 5)' + ba^+nbp?) 
It follows from (3. 30) that the coefficient of the second sum 

may be approximated by {l/b(l. 5)'), so that the exponent of 
(3. 29) is almost exactly equal to: 



(1.5) " b(1.5) 



(3.31) 



A similar sort of simplification of the coefficient of £ 

in (3. 29) may be made from a consideration of the relative 

2 2 2 

magnitudes of nbp and ba . The value of a represents the 

variance of a quantity a, idealized relative intensity; and 

2 
p is, according to assumption 5) of Section IV, the variance 

of small purturbations of the latter. It is reasonable to assume 

2 2 2 2 

that a » p , and that even ba >> nbp . By this approximation 
n n n n 
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it follows that the coefficient in (3, 29) is close to: 

P(CL) . (3.32) 

Vbo-„( ^2^1. 5)"" 
If this and the approximation in (3. 21) are applied to (3. 29), and 

similar approximations are applied to corresponding formulas 

for CE{J^) and CH(J^); Formula (3. 13) reduces to: 

J J 



Q*(jJ)=: 



1 SL(.1) 1 SE(j) {3.33) 
P(CL) o 2(1- 5)' I P(CE) r, 2(1. 5)^ 
Vba, ^ ^Vb'g; ^ 

1 SUi) 1 SE(.1) i SH(.1) 

P(CL) ^ 2 (1. 5)2 . P(CE) ^ 2 (1. 5)2 . P(CH) ^2 (1. 5)' 
VF^ ^ ^ Vb'a; C ^ Yb"a',' t 

Where: o., a\, and a'-j| are respectively the idealized relative 

amplitude variances of elements of CL, CE and CH; 

b=;[:(ub^ (3.34) 

i m 

and (U , . . . , U ) is some paradigm intensity 

pattern for component regions of members of CL; 



b'=E(U'b^ (3.35) 

t 
and (U' , . . . , U' ) is some paradigm intensity 

profile for component regions of members of CE; 

b"=I;(U"*)^ (3.36) 

t 

and (U" , . . . , U"™) is some paradigm intensity 
profile for component regions of members of CH; 

93 



M .\^lh (3.37) 



SL(3) = (^uVj^^) /b 

SE(3) = (:^U'VJ^3^)^b' (3.38) 

SH(i) = (|:u"V ^)2/b" (3.39) 

1 m 1 

arid (v. ^ , ... , V. J are the intensities m J-. 

It is necessary to assign values to the remaining 
constants in (3. 33). First we shall assume that lines and 
edges are equally likely, so that P(CL) = P(CE). Also in 
the examples which follow, it will become clear that an edge 
or line occurs in one sample out of 75, so that: 

1/75P{CH) - P(CE) = P(CH). 
Secondly, it follows from empirical investigation of the 
range of spread of homogeneous samples with various 
gradients, that five is a reasonable value for b"a!j| . It was 
already pointed out that 30 is a reasonable value for ba^^, 
and we shall assume this is the case for b'a' as well. Thus: 

Q*(jJ) 

SU1) SE(.1) 

2(1.5)' 2(1.5)' 

e e 

(3.40) 



SL(.1) SE(.1) SH(.1) 

2(1.5)2 2(1.5)' 2(1.5)2 

e e ^^^°^ 
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For values greater than . 5 this function is nearly 
monotone with: 

Max(SL(j)/2(l. 5)^, SE(j)/2(l. 5)' ) - 

SH(j)/2(l. 5)^^ - Ln(450. ), 

or equivalentlyj with the function Q**(Jj) given by: 

Q**(J^) = 
J 

Max(SL(j), SE(j)) - SII(j) - 27. 5 (3. 41) 

This follows from an observation about the results given later 
in this section, namely that SL(j) and SE(j) are rarely of the 
same order of magnitude. This is not surprising, in view 
of the fact that one expresses the similarity of the obtained 
pattern to a noise free edge, and the other expresses the 
similarity to a noise-free line. For values of Q*(J.-) 
greater than . 5, the pattern must be at least somewhat 
similar to either an edge or a line. It could not be 
simultaneously similar to both. Consequently the formula 
(3. 40) above, which is of the form 

A + B 

A + B + C 

has the property that when its value is above . 5, A and B 
are considerably different in value, and both positive. 
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It is not difficult to show that in such a case if, for example 

Max(A, B) > 10 Min(A, B), 

then the approximation: 

Max(A, B) ^ A + B 
Max(A, B) + C ^A+B + C 

is good to better than 6 percent. The fact that Q*(jb is 

monotone with Q**(J.) follows from the fact that 

J 

Max(e^ e^) 
Max(e^, e ) + e*^ 

is monotone with Max(a, b) - c, if a, b and £ are positive. 

It is desirable to apply the predicate to a variety of 

regions which are homogeneous, contain off-center lines, or 

contain centered lines or edges. We have chosen data from a 

series of areas on the visual field which are 200 units wide 

and contain the images of vertical edges and lines approximately 

centered laterally. The predicate will be applied to 75 

successive overlapping regions 50 units wide, which are 

offset relative to each other by two units and thus exactly 

cover the 200-unit-wide area. The procedure will be applied 

to vertically oriented regions only, since only vertically 

oriented lines exist in the data samples. The fact that 75 
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regions are tested and only one contains a centered line or 
edge justifies the relation P{CH) = 75P(CE) used above. 

Finally, before applying the predicate to real data, 
which will be done by computing the function Q**(J .), it is 

J 

necessary to give values for the sets | U }, | U' | and |u" } 
used in formulas (3. 34) - (3, 39). It will be recalled that 
these are paradigm sets of intensity values at points in some 
fixed geometry within a square sub -region. They correspond 
respectively to a line centered in the region, an edge centered 
in the region, and a homogeneous region. According to the 
fixed geometry which we shall use, the points will be equally 
spaced along a line traversing the square sub-component 
through its center and oriented normal to the expected line. 
These values may be ordered in an obvious manner according 
to their positions along the line mentioned. They are shown 
graphed as a function of this ordering in Figure 3. 6. The 
values of b, b' and b" for these sets of intensities are all unity. 
The intensities from the real world will come from single scans 
of 100 points taken normal to a vertical edge or line in the 
real world. The points will be separated laterally by two 
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units, so that the total width of the area scanned will be 200 
units. If it is desired to apply Qi'*(J.) to some region along 

J 

this scan, out of the 100 intensities I., ... , Ii qq, some 

^^^ ^1+1' • • • ' ■'■i+25 ^^^ within the region. Then it is 

trivial to select intensities from among these to be used as 

the values v- -,,..., v . ^ for use in formulas (3. 37), 

(3. 38) and (3. 39). They are simply I , . . . I. „p. respectively. 

Thus we will be applying (3. 41) to 75 successive 25-tuples from 

the 100 intensities in a particular scan. 

It is important to digress here to point out the 
advantages of the sort of geometry of points within a sub -region 
which is used above. This advantage will be realized in the 
next chapter when it is desired to apply the regional predicate 
exhaustively over the whole field. In particular it will be 
desired to compute the predicate for lines of all orientations 
through any point P on the visual field. Consider those 
situations where the regions are oriented within + 45 degrees 
of the vertical axis of the vidissector, and all pass through 
some point P; and consider only the square sub -region 
containing the point P, which will be termed the p-th 
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sub -region. We shall in chapter IV take intensities relevant 
to regions so oriented, along lines separated by approximately 
the width s^ of a region, and oriented parallel to the x-axis of 
the vidissector. We may thus assume that there will be a 
unique scan passing within s/2 of the point P; and that the 
intensities I., ... , I along its length within a distance s/2 
of P are exactly the ones from which the v- will be selected. 
The procedure for selecting the values v- from among this 
set is parameterized by the angle which the region makes 
with the vertical. If the spacing of points in the fixed 
geometry of points is d, then one wishes to choose points from 
among the I. , ... , I such that their actual spacing is 
approximately d/Cos(0). If the spacing of the points along the 
scan line is itself d, which will usually be the case in practice, 
then this matching of obtained intensities with those in the 
prototype amounts to "stretching" the former by some factor 
between 1 and •^. Thus the various values of, e. g, , 
SE(J-i „) = 2^ V. U , which are used in the computation of 
Q*(J-j ) for various regions containing P, differ only in the 
amount of "stretching" of one spatial axis relative to the 
other in the matching of intensity values of the scan against 
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those of the set \ U }. Denoting these various values of 

CE(J- ) by CE(J. , 0), it turns out empirically that the range 
J* P ' J» P 

of these values is rather small; and in particular for | j <45°, 

the values are within something like 2 percent of the closer of 

CE(j" , 10°), CE(J^ , 30°), One may thus compute these two 
3' P J,P 

values only, and for any region containing the intensities from 
which they were computed, use one or the other in the 
calculation of Q*(J-j ) for that region. In fact we shall in 
chapter IV actually compute these two values by stretching the 
spatial dimension of the paradigm, rather than vice versa. 
This amounts to using two peaks of different widths as paradigm 
profiles for lines, and two edges of different severity of slope 
for paradigms for edges. It should be clear that if the fixed 
geometry of the points were two-dimensional, e. g. on a 
square grid, no such simple accommodation for the various 
angles might be possible; and doubtless the amount of 
computation involving the intensities in each neighborhood 
would be far more than the calculation of two values. 

To return to the application of Q-!^* to real data, our 
first example, in the upper left of Figure 3. 7, comes from 
a horizontal scan across the forward edge of a cube. The 
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scan covers approximately one eighth of the apparent width of 
the cube. The cube is a solid metal block which has been 
painted with white paint so as to have a very uniform surface 
and sharp edge. In each of the eight figures, the logarithms of the 
reciprocals of intensity are given as the bottom of the two curves. 
The absolute range of values is 11 units, where 64 units corresponds 
to a factor of two in intensity. It is easy to infer from the physical 
properties mentioned that the slope of the intensity curve on either 
side of the edge is due to a systematic non-uniformity in illuminatiori, 
rather than to a non-uniformity of reflectance properties. At the 
edge itself there appears to be a sort of "negative highlight"(a 
local peak). This appears also at the centers of the scans in the 
upper right and upper-middle left examples of Figure 3.7, which 
were taken from the same cube; and is apparently a property of 
this particular edge. The 100 intensities in the example give rise 
to 75 values of Q** which are graphed above the intensity curve. A 
value of Q** is derived from 25 successive intensities consisting of 
the one just below it and twelve on either side. Thus a peak in the 
value of Q** should exactly coincide with a peak or discontinuity 
in the intensity profile. The horizontal lines in each of the 
eight figures are placed at distances along the Q** axis which 
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correspond to values of Q* which are indicated on the lines. 

A reasonable threshold value for Q* appears from an 
analysis of this example, and those appearing later in this 
section, to be about 0. 1. A cutoff level of approximately this 
magnitude will be used by the program described in the next 
chapter. This cutoff value is chosen on the basis of its giving 
about 70% false positives. There is no simple exact relationship 
between the Q* cutoff and the false positive error rate; however 
it follows from the first theorem of chapter two that if /3 is the 
cutoff value for Q*, then 1 -/3 is an upper bound of the false 
positive error rate. This may be seen by noting that for the 
threshold procedure, the sum in (2. 3) gives the false positive 
error rate. Every element in the sum is some probability 
P(J.(S)) times a value less than 1-/3. Since the values of 
P(J(S)) sum to one, the sum in (2. 3), i. e. , the false positive 
error rate, is less than 1-/3. Thus thresholding Q* at 0.1 should 
give a false positive error rate less than 0.9. However, the 
extent to which the false positive error rate is actually less than 
0. 9 is an empirical matter. Evidently the various probabilities 
and so forth are such as to give an actual value of 0. 7 in this 

case. 

104 



An obvious question now arises: how subtle may an 
edge or line be relative to the noise level in order to be just 
detectable, e. g. , with the threshold of 0. 1. An example of 
such an edge is given in Figure 3. 7 at the upper right. It is 
possible to make a general quantitative description of such a 
line or edge by means of a value derived from the cutoff value of 
Q**. Suppose the cutoff value is 0. 1; then we have: 



or: 



or 



or: 



or: 



Max 


r SUi) SE(.i)l 
2(1,5)^ p2{1.5f 




Max 


r SUi) sE(i) ■ 

2{1.5f 2(1. 5)^ 


SH(.i) 
^ 2(1.5)2 
+450 C 



« 0.1, 



9. Max 



SUi) SE(.i) ] SH(.i) 

2(1.5)' 2(1.5)' 2(1.5)= 

t , t I « 450. t 



Max(SL(j)/2(1.5)'SE(j)/2(1.5)i^+-Ln(9)*» 
SH(j)/2(1.5)'+ LN(450. ), 

Max(SL(j), SE(j)) - SH(j) - 27. 5 == 2(1. 5)'Ln(9) = 9.9 

Max(SL(j), SE(j)) - SE(j) « 17. 6 (3.42) 
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Since SL(i), SE(j) and SH(j) are positive, the minimum value 
of Max(SL(j), SE(j)) subject to (3. 42) occurs when SH(3) = 0. 
This is the case when the net intensity gradient for the sample 
ife zero; a fact which can be deduced from (3. 39) and a 
knowledge of some of the arithmetic involved with simple 
least-squares analysis. In this case for a marginally 
detectable line or edge, SL(j) or SE(j) has the value 17. 6. 
If the intensity profile follows the form, e.g., of the upper 
curve in Figure 3. 6; then it is not difficult to see, by the use 
of (3. 37), what its actual peak to valley range would be: 

Suppose the 25 intensities in the noisy intensity profile are 

1 25 

given exactly by aU , . . . , aU . Then SL(j) for these 

values is, according to (3. 37): 

E (U^)2 

However, the denominator term, which is simply b as defined 

in (3. 34), was chosen to be unity for the set of U 's in 

2 

Figure 3. 6; so the value of SL{j) turns out to be simply a . 

This, it was agreed, is 17. 6; so the value of a is 4, 2. 
Thus a marginally detectable intensity profile of this shape 
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is similar to that at the top of Figure 3. 6, but multiplied by a 
factor of 4, 2; and thus has a peak to valley range of about 
three units. In the case that SE(j) is a maximum and the 
phenomenon is edge-like, one may assume the approximate 
shape of the intensity profile is that given by the middle curve 
of Figure 3. 6. By a similar argument, the marginally 
detectable sample of this sort has a range, in this case net 
intensity difference, of about 1. 7 units. The central feature 
of the scan in the upper right of Figure 3. 7 has a value of 
Q* at threshold; and the amplitude of the peak at the center 
may be compared with the former of the two theoretical 
values just referred to. The intensity range for the whole 
scan is 17 units, from which it is easy to see that the peak to 
valley height of the small peak in the center is about four units. 
This compares well with the theoretical value of three units. 
Clearly the amplitudes of marginally detectable lines 
and edges must be considered in relation to the noise level. 
Noise may in general be due both to time noise, random 
fluctuations in intensity measured at a point under constant 
illumination; and to space noise, which consists in random 
fluctuations in intensity as one scans across a surface, A 
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combination of the two may be obtained from an analysis of 
successive differences in intensity of the above samples in 
areas away from the known features. This analysis yields a 
standard error of succeissive differences of about two in all 
cases. Since the variance of successive differences is twice 
the variance of individual values about a mean, then the 
standard error of the latter should be about 2/^2^, or about 
1. 6. This is approximately the value of the standard error 
involving time noise only, as reported in (3. 5). Consequently, 
the space noise is in this case negligible, which is not 
surprising in view of the deliberate choice of smooth-surfaced 
objects. In any case, the marginally detectable amplitudes 
of 3 units and 1. 7 units for lines and edges respectively 
should be thought of in relation to an underlying noise level 
whose standard error is 1.5. For example, using the 25 
point geometry previously described, one may state that a 
marginally detectable peak has a peak to valley range of 
twice the noise level, and a marginally detectable edge has a 
net intensity difference of about 1. 2 times the noise level. 

Six additional examples are given in Figure 3. 7. 
The left-hand upper-middle scan was taken from the same 
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cube and under the same lighting conditions as the top two in 
the illustration. The other five were obtained from a scan 
across the vertical forward edge of a second cube under 
various lighting conditions. The lower-middle pair, marked 
NEDGE2A and NEDGE3A, were taken under identical lighting 
conditions, but at different heights along the vertical edge. 
This cube is uniformly painted, but the edges are slightly- 
rounded, accounting for the pronounced highlight (dip in 
intensity, since the values are actually Log(l /intensity)) 
apparent in some of the profiles, particularly the one marked 
NEDGE2A. 

There is a certain weakness in the use of so great a 
region width as 50 units, namely that the intensity profiles 
across a region may have non-negligible gradients toward 
the edges of the region, appearing, e, g, , like the profiles 
in Figure 3. 8. For this reason, it is useful to generalize 
the thresholding procedure so far described by computing 

values SL^(j) SL^(j), SE^{j), . . . , SE^(j) for a set 

of n peaked profiles and m edge profiles, and using a 
maximum of all these values in place of the maximum over 
two elements as in 3, 41. Denoting this generalized version 
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of Q** by Q'**, we may express it in two slightly different 
forms: 

Q'**(Jj) = 

Max (SL.(j),SEj^(j)) - SH(3) - 27. 5 = 
i,k 

Max (SL.(j) - SE(j), SE, (j) - SE(j)) - 27. 5. (3. 43) 

i,k 

This generalized formula will be used in the program described 
in the next chapter. In the remainder of this chapter we will 
discuss its application to the previously analyzed data of 
Figure 3. 7. 

For present purposes, we shall consider only a possible 
multiplicity of paradigm peak profiles. Four additional 
paradigm profiles besides the one in Figure 3. 6 are illustrated 
in Figure 3. 8, and denoted by WPEAK, CPEAK, BPEAK, and 
APEAK respectively. It is instructive to compute individually 
the values whose maximum gives the value of Q'** according 
to the second expression in (3. 43). In particular we shall 
consider six values in the following order: 

SL (j)-SH(j), corresponding to the paradigm WPEAK; 

SL2(j)-SH(j), corresponding to the paradigm CPEAK; 
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SL (j)-SH(j), corresponding to the paradigm BPEAK; 

SL.(j)-SH(j), corresponding to the paradigm APEAK; 

SE(j)-SH(j), corresponding to the edge paradigm of 

Figure 3. 6; 

SL (j)-SH(j), corresponding to the peak paradigm of 
5 

Figure 3. 6. 
In Figures 3. 9 and 3. 10, we have these six "components" of 
Q'^!'* individually graphed above the corresponding intensity- 
profile, in the above order top to bottom, in a manner similar 
to that used in Figure 3. 7. 

One may make a series of observations concerning the 
various curves in Figures 3. 9 and 3. 10. Comparing the 
lowermost two component curves in the upper left of Figure 3. 9, 
it is easy to see that the pair of peaks in the upper left 
illustration of 3. 7 were due to the peak-detector profile 
entirely. This follows from the fact that the peaks appear 
only in the lower of the pair of curves from 3. 9 just mentioned. 
Regarding the same profile, one m^ay see that the major peak 
from among the component curves is the uppermost. This 
corresponds to the profile WPEAK, which is a wide peak. 
Evidently the feature represented by the maximum of this 
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curve is the inverted wide peak to the left of the center of the 
intensity scan. Another observation concerns the lower right 
example in 3. 9. It is reassuring to note that although the 
discontinuity present in the intensity profile is quite large, 
it was "noticed" by none of the peak-detector profiles. This 
is evidenced by the fact that in the top four component curves, 
and in the bottom one, tne curves are not above threshold in 
the vicinity of the extreme intensity discontinuity. On the 
other hand, the next -to-bottom curve, corresponding to an 
edge, has an extreme peak at this location. Lastly, one may 
make the general observation that the maximum of all six 
component values of Q'** may be thresholded at a considerably 
higher level than Q**, for example at a value corresponding 
to aQ'* value of . 99, and still detect all edges, but at a 
considerably better signal-noise ratio. Observation of the 
figures shows that at this level a few false positives remain, 
e. g. , the peak to the right of the large discontinuity in the 
lower right example in Figure 3. 9. An examination of the 
intensity curve itself would appear to justify the contention 
that there really was "something there" after all, perhaps 
a spot on the cube, or an insensitive spot on the vidissector 
photocathode, 
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CHAPTER 4 



A COMPUTER PROGRAM FOR FINDING LINES 



INTRODUCTION 






Figure 4. 1 
Figure 4. 1 illustrates tiie first stages in the analysis 
of scenes by the program to be described in this chapter. On 
the left is a photograph of a set of objects placed before a 
random access optical input device connected to a digital 
computer. Programis in the computer have access to 
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intensities in the field of view at arbitrary locations on a 
2OOOO0 by 2 0000g grid. Intensities are obtained by the program 
along 50 vertical- and 50 horizontal scans of 500 intensities each. 
The intensities are converted into "feature points", as 
illustrated in the center illustration of 4.1. Feature points are 
locations where the scans are adjudged, by spatially local 
processing, to intersect with an edge of an object in the real 
world. Often noise in the real world or within the vidissector 
will produce an intensity configuration along part of a scan 
which is edge-like and gives rise to a "false positive" feature 
point. The right-hand portion of Figure 4.1 illustrates the 
output of a program which extracts lines from an array of 
feature points and which is designed to ignore these 
extraneous "false positive" points by detecting chains of 
lined-up feature points. A final stage of the analysis, not 
illustrated in Figure 4.1, is the proposing and verification 
of lines not located by the above procedure. Lines are proposed 
on the basis of lines already located, in places where figures 
appear to be incomplete. 
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II. A DESCRIPTION OF SCENES ANALYZED 

In this chapter we will describe an object recognition 
program based on the theory so far developed. This section 
will be devoted to a discussion of the restrictions imposed 
by the theory on the scenes analyzed, and on the optical input 
device used. Various scenes conforming to the restrictions 
will be illustrated. 

The theory developed in the preceeding chapters is 
applicable to certain types of scenes, and to optical input 
devices satisfying certain conditions. Stated informally, the 
restrictions on the scenes are: 
1.1) The scene consists entirely of relatively homogeneous 

regions bordered by edge lines. 
1. 2) The edge lines are all straight. 
1. 3) The objects of the scene may be recognized entirely 

from the edge lines. 

The first of these restrictions is fundamental. It is 
virtually equivalent to the assumption that an intensity profile 
across a scene consists of a series of smooth curves bounded 
by discontinuities as in Figure 4.2. 
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INTENSITY 



HYPOTHETICAL INTENSITY PROFILE ACROSS A 
SCENE IN THE ABSENCE OF BLURRING AND NOISE 



Figure 4. 2 
The second of these assumptions is essential if one is 
applying an edge test only to narrow rectangular regions, as 
is the case in the program under consideration. Of course 
one could develop a theory, analogous to the foregoing, for 
amalgamating local feature point information over curved bands, 
and thereby detect edge lines of arbitrary shape. However we 
have not done this; it almost certainly would require computation 
of an order of magnitude above that used in the current program. 

The third restriction is a matter of convenience. 
Actually the program may be presented with scenes not 
conforming to this restriction without any impairment of its 
behavior. For example, it might be presented with an edge -on 
view of a cylinder, and successfully recognize the outline. 
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The roundness, though detectable by the vidissector from the 
intensity gradient across the surface, would not be detected by 
the present program. 

These restrictions are summarized in Figure 4.3. 

NOT APPROPRIATE: 





APPROPRIATE: 





ATTENTION WILL BE LIMITED TO A 
CERTAIN CLASS OF SCENES 

Figure 4.3 
The theory requires certain restrictions on the 
instrument which obtains the intensities from the real world. 
These are, in summary: 
2. 1) The extent to which the input device blurs the image 

received must be uniform over the whole field. 
2.2) The extent to which the intensities are modulated by 
noise must be uniform over the field, and independent 
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of relative intensity. 

Stated more specifically, the first assumption says 
that if I(x, y) represents the intensity of the scene over the 
field of view, D; then the intensity received by the optical 
input device in the absence of noise is: 

I-(x, y) = y I{x-u, y-v)f(u, v) dudv 
D 

for some blurring function f(x, y). 

These assumptions are summarized in Figure 4. 4: 




ACTUAL INTENSITY 



INTENSITY VALUES 



FROM OPTICAL 



INPUT DEVICE 



BLURRED INTENSITY 



BLURRED INTENSITY 
WITH NOISE 



THE INTENSITIES OBTAINED FROM THE 
OPTICAL INPUT DEVICE ARE UNIFORMLY 
SUBJECTED TO BLURRING AND NOISE 



Figure 4. 4 
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Some scenes analyzed by the program are shown in 
Figure 4. 5. 

It is easy to see that criteria 1. 2 and 1. 3 are 
satisfied by scenes of this sort. However, it is not readily 
apparent that criterion 1. 1 is satisfied. This follows from 
the fact that real lines such as highlights and cracks between 
cubes, in the absence of blurring and noise are never of zero 
width as stipulated in the definition of a line. However it is 
easy to see that they may be of nonzero width so long as they 
are sufficiently narrow to be indistinguishable from zero -width 
lines under the blurring function. This blurring function was 
discussed in Section H of chapter three. From its properties 
it is easy to see that blurred versions of lines of less than 
about five units width, where one unit is 1/2000 of the 
vidis sector field width, are virtually indistinguishable from 
each other and from blurred versions of lines of negligible 
width. Criterion 1. 1 has thus been satisfied for these figures 
by keeping the apparent widths of the edge lines below the 
aforementioned five units. 

Criterion 2. 2 has been met by the optical input device 
used, an Information International vidissector, by the nature 
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EXAMPLES OF SCENES ANALYZED 



Figure 4,5 
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of its design. However certain care is required to satisfy 2.1, 
and it can not be completely satisfied by the instrument used. 
The blurring inherent in the vidissector is a function 
of optical focus, of internal electronic focus, and of the size 
of the internal aperture used to collect intensity data from a 
point in the visual field. The latter two matters have been 
discussed at length by Horn (Horn 1969), £ind in the literature on 
the instrument itself. Suffice it to say here that there is a 
more or less uniform degree of blurring inherent as a result 
of these two factors, though a certain amount of non -uniformity 
is detectable by careful measurements. It is fortunate that the 
order of magnitude of these effects is somewhat greater than the 
optical blurring of the system when it is focused to miinimize 
this blurring. By keeping the objects in the field of view in a 
sufficiently flat plane, it is possible to have the non-optical 
blurring effects dominate the optical effects even for portions 
of the field which are relatively out of focus. This, to a first 
order of approximation, insures a uniformity of over-all 
blurring, despite the fact that the purely optical blurring is 
quite non-uniform. The criteria for "a sufficiently flat plane" 
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for the image is given by laws of optics. From the well known 
relation that: 

1/F = l/a+ 1/b 
Where: 

a = object distance 

b = image distance 

F = focal length of the lens, 
it follows that: 

da/db = - a^/b^. 

Hence for a given displacement Aa of the object from perfect 

2 2 

focus, the image displaces by an amount -(Aa)(b /a ). This 

quantity, divided by the focal ratio f, gives the amount an image 
is blurred by displacing the object a distance Aa from perfect 
focus. Thus the allowable scene depth for a particular 
maximum allowable amount of blurring depends on the square 
of the scene-distance image -distance ratio; and inversely on 
the f-ratio. If a particular object or arrangement of objects 
increases in absolute size, and its distance from the lens 
increases proportionately (and if the object distance is large 
relative to the lens -image distance), its image remains 
approximately constant in size, but the depth of the object 
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remaining in good focus increases linearly relative to the size 
of the object. It is thus possible, for a given arrangement of 
objects to keep the blurring over the whole field down below 
any pre-assigned limit by making the f-ratio sufficiently large, 
and by making the objects sufficiently large. In practice, due 
to the low sensitivity of the vidissector it was often impossible 
to insure adequate depth of field by the use of a suitably small 
aperture alone. In particular, in the case of the objects 
illustrated in Figure 4. 5, it was necessary to make an effort 
to make the objects as large as possible to satisfy criterion 2, 1. 



III. THE DETERMINATION OF FEATURE POINTS FROM 

INTENSITY INFORMATION 



In this section we will be concerned with a discussion of 
a procedure for extracting information from a set of intensities 
taken over a visual field, as a first step in the determination 
of the locations of lines in the field. The relevant theory, 
developed in the previous chapter, will be reviewed; and an 
algorithm for reducing intensity information to "feature points" 
will be discussed. 

In applying the line predicate to various regions on 
the visual field, we shall obtain intensities along scan lines 

126 



perpendicular to the axes of the vidissector. In particular we 
shall use intensities obtained from scans parallel to the x-axis 
in applying the line predicate to long rectangular regions 
oriented within + 45° of the y-axis. Similarly, whenever a 
line predicate is applied to a region within + 45° of the x-axis, 
intensities from scans parallel to the y-axis will be used. We 
may thus henceforth without loss of generality assume that we 
are dealing with regions oriented within + 45° of the y-axis, and 
with intensities obtained from scans taken parallel to the x-axis. 
A scan shall consist of 500 intensities tsiken at intervals of 
2 units along a line, where, as in the previous chapter, one 
unit is 1/2 000 of the full field width. The line is centered 
laterally in the field. Fifty scans will be taken at intervals of 
20 units, with the 25-th scan half way up the field of view. The 
set of scans will thus encompass a square area exactly centered 
in the field of view, whose edge length is 1000 units, or one 
half the edge length of the entire field of view. The reason 
for not using the entire field is simply that there are various 
physical problems at the edges, one of which involves an 
aperture which prevents obtaining intensity values at the corners. 
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The central square used is approximately the largest useful 
square region in the field of view. For purposes of visualizing 
the following explanations, it may be assumed that an entire 
set of 2500 intensities is obtained at once, prior to any further 
processing. This is not entirely true, but may as well be 
considered to be the case. 

We will want to be able to apply the line predicate to 
any of approximately fifteen million rectangular regions in the 
field of view. This figure is obtained from the fact that we 
are considering regions extending between pairs of scans, and 
the latter number 50.'49/2 = 1250; we are considering regions 
at any of fifty orientations; and we are considering regions 
with any of 500 different intercepts. In fact we shall apply a 
modified version of the predicate to all of a subset of these, 
namely those which traverse the field entirely, which number 
50* 500 = 25, 000. This will be discussed in Section IV. Also 
the predicate will be applied to shorter subregions of those 
among the latter which appear to contain a line somewhere 
along their lengths. Further the predicate will be applied to 
regions proposed on the basis of lines already found. This will 
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be discussed in Sections V, VI and VII. 

The application of Q* to a region on the field of view 
consists in computing the value given by (3. 13): 

QHJp = 

CL(jJ) + CE{jy) 



CLCjJ") + CE(J^) + CH(j") 



where CUj']) is given by (3. 2 9), and CE(J^) and CH{J^) are 
given by similar formulas. A region whose maximum minus 
minimum y-distance is s^ units intersects with lls/20llscan 
lines, where Hill means the closest integer to L Instead of 
assuming that a region is of a particular fixed width measured 
perpendicularly to its length, we shall assume it has a fixed 
x-direction width of 50 units. Consequently the intensities 
along a particular scan which lie within a rectangular region 
will be exactly 25 in number, since the separation of intensity 
points along a scan is two units. We shall also assume that 
the length of a sub -region, instead of being a fixed quantity 
as measured along the axis of a region, has a fixed y-direction 
length of 20 units. Also sub -regions will be taken to be 
vertically centered on a scan line. This arrangement is 

129 




diagrammed in Figure 4. 6. 
SUB -REGI0NS''^K"^^ — ^/ 7fo 

1" 

SCAN 
LINES 

SUB -REGION ^"Sj^T — / RECTANGULAR REGION TO 

SCAN 

SEGMENTS /— ^ WHICH PREDICATE IS APPLIED 

Figure 4. 6 

We may thus identify a segment of a scan enclosed within a 

region with a particular sub -region. The advantages of this 

sort of geometry were discussed at the end of the previous 

chapter. We shall use a specific term for the segment of a scan 

which we are associating with a particular sub -region, namely 

a sub -region scan segment. In summary, a region of length 

a consists of lla/20|| = m_ sub -regions; and the value of Q* is 

computed from the m sub -region scan segments by calculating 

values of CL, CE, and CH according to formulas of the form 

given by (3. 29). 

Evidently the intensities of a particular sub -region 

scan segment enter into the computation of Q* for several 

thousand regions enclosing the segment. Examining (3, 29) 
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it is apparent that, e. g. , in computing CL, it is necessary to 

compute a value R. from the intensities in this segment, and 

also a value a* ; and that CL is determined entirely from these 

values. It is not hard to see that these values are the same for 

a particular segment, when the enclosing regions are all at the 

same inclination. Further, it was pointed out in some detail 

in the last chapter that these values for a particular segment 

do not vary a great deal with orientation of the enclosing 

rectangular region assuming, as we are, that the regions 

remain within + 45° of a normal to the scan lines. Thus, as 

before, only values of R. , etc. , for two region orientations, 

10° and 3 0° , need be computed. This is done, as in the previous 

chapter, by using two paradigm profiles of different widths. 

Finally, we may recall that at the end of the previous chapter 

the procedure of computing a single value of R, R', etc. , for 

a particular segment and orientation was generalized to the 

computation of several, corresponding to different possibilities 

of the shape of the intensity profiles toward the edge of the 

region. Consequently, for a given sub -region scan segment, 

we will compute exactly the values R. , , . . . , R , 

^ J,k, 1 ],k, 6 

a* , , . .. , a* , R' ,,•■•, R! , _, a'* , ... , 

3,k, 1 j,k, 6 3.k, 1 j,k, 5 3, k, 1 
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a'* , and also R" and a"* . The six E's and a*'s are 
j,k, 5 j,k j,k 

computed as in (3. 25) and (3. 23) from six different peaked 

1 25 1 25 

profiles (U. , . . . U^ ),..., (Ug, ... , Ug ) of various widths 

and with various combinations of gradients towards the ends. 

Similarly the R"s and a'*'s are computed from five different 

cliff -like profiles. Clearly, no such multiplicity is necessary 

in the case of R" and a"*, which corresponds to a homogeneous 

region. These paradigm profiles, 2 5 -tuples of values (U^ , 

25, 
. . . , U. ), etc., from which the R's, a*'s, R"s, a'*'s, and 

R" and a"* are computed, are illustrated in Figure 4. 7. 

Since values of R. , , , at , ,,..., R' ^, a'* , 
J,k, 1 j,k, 1 J,k, 5' j,k, 5 

R'.' , , and a'.'* will be required for every sub -region scan 
J» ^ Jik 

segment, they are all computed. Since there are 475* 50 
sub -region scan segments, and 24 values computed per segment, 
this is a matter of 570, 000 intermiediate values, an order of 
magnitude more than the 25,000 intensity values from which 
the values were computed. However, the derived set of 
values admits of considerable compactification. First, for 
a particular sub-region scan segment, it is necessary to retain 
only the minimum of all the R's and R"s, together with the 
corresponding value of a* or a'*. Secondly, consider the 
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values of R" minus this minimal R or R' for the 475 successive 
sub -region scan segments along a particular scan. Then there 
exists a threshold for these differences such that most values 
lie below the threshold and indicate with virtual certainty that 
a line does not traverse a region containing that segment. 
Such values may be omitted, thereby indicating that they fall 
within this category. Also, only the local maxima of the values 
above threshold need be considered. Consequently an entire 
collection of 475 sets of values may be reduced to somewhere 
around ten sets. These latter sets need only consist of 4 
values: the value of R" minus the minimal R or R' which is 
locally maximal; the value of a* or a'- corresponding to the 
minimal value of R or R'; an indicator as to which R or R' 
was minimal; and the x co-ordinate of the point in question. 
Such a four-tuple, together with the y co-ordinate of the scan 
from which it was derived, will be termed a feature point. 
We will not justify in detail here why the 570, 000 values can 
be reduced to approximately 500 feature points. However we 
shall point out that the subsequent use made of derived 
information in the computation of Q^= and similar values uses 
no more than the reduced feature point data; that many of the 
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values can be considered to be in a single null category; and 
that the remaining values have sufficient regularity to be 
specifiable in terms of their local maxima only. 

The program (GETFP X S) obtains intensities for a 
single scan and performs the calculations sketched above. 
The second argument is 1 or depending on whether a scan is 
to be made vertically or horizontally respectively. The value 
of X is the appropriate ordinate or abcissa on a scale from 
to 500. Five hundred intensities are taken across the field 
along the line defined by the arguments and are processed 
according to the algorithm sketched above. The output is a 
series of 4-tuples as described above. A schematic diagram 
of this process appears in Figure 4. 8. 

In the line finding program which has been developed, 
the sub-program GETFP is called 50 times for vertical scans 
and 50 times for horizontal scans. This procedure is illustrated 
in Figure 4. 9. In this figure, the short lines indicate the 
locations of maxima previously described. Their lengths are 
proportional to the corresponding values of R" - Min(R, R') 
for small values of this difference. For values of this difference 
above a certain threshold, a constant length of line is used 
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corresponding to this threshold value. These points will be 
termed feature points. Rasters of feature points corresponding 
to the illustrations in Figure 4. 5 are given in Figure 4. 10. 



IV. A GLOBAL PROCEDURE FOR EXTRACTING LINES 
FROM ARRAYS OF FEATURE POINTS 



If one were to examine the feature point rasters of 
Figure 4. 10, one would be able to pick out almost all the lines 
in the figures they represent. The perception of some of the 
lines may depend on complex global perception processes, 
for example, the use of obvious lines to provide clues as to 
the existence of less obvious ones. A computer approximation 
to this process will be the subject of subsequent sections. On 
the other hand, some of the lines are obvious without either 
reference to other parts of the figure or a priori knowledge 
of what constitutes a plausible figure. This is because the feature 

points are both strong (represented in 4.10 by long lines), and 
they line up very exactly. This section describes a program, 
LINES, which takes advantage of the obviousness of certain 
lines to find them in an array of feature points, by a rather 
simple and not overly lengthy procedure. 
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The problem of locating lines by accumulating locally- 
obtained information has been approached by line following 
procedures. (Roberts) This often runs into the problem that 
the line follower gets lost, since, in effect, it is making a 
series of decisions (whether or not to continue the line) on 
the basis of local considerations. This problem becomes 
greater as the number of lines in the scene increases. Line 
followers are also subject to the problem of never having 
"attached" themselves to certain lines in the first place. It 
is thus desirable to use a more exhaustive and global procedure, 
for example, covering the feature point raster with a very large 
number of narrow rectangles, and applying a thresholded 
predicate to the feature points within each rectangle. The 
apparent tremendous cost of such a procedure can be greatly 
reduced by observing that if a certain narrow rectangle contains 
evidence of a line, then so does a rectangle with the same 
orientation but extending across the entire field. The converse 
of this statement is not true, since a rectangle across the 
whole field may contain a large amount of line evidence in the 
form of scattered noise and spurious values due to feature points 
along lines intersecting the rectangular region at some angle. 
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However, the testing of narrow rectangles across the whole field 
does provide a sort of screening procedure. Any relatively- 
strong lines will fall within at least one of these bands, and thus 
be detected. On the other hand, it is necessary to further analyze 
the contents of one of these bands which has a large amount of line 
evidence in it in order to determine both if there really is a 
line within the band; and if so, where it is. 

The procedure for assessing the total amount of evidence 
for the existence of a line within a band is based on the idea of 
projecting a two dimensional array into a one dimensional line. 
For example, if an array of points (x^, y^ is given, and it is 
desired to determine the locations (x-intercepts) of vertical 
bands containing a large number of points, then it is merely 
necessary to histogram the x-co-ordinates of all the points. 
The resulting histogram has large values for x-intercept 
values of bands containing a large number of points. By this 
procedure, all vertical bands are investigated in parallel. 
The procedure is simply modified for arbitrary orientations, 
by histograming numbers of points versus a quantity 
X. + ay., where a depends on the orientation under consideration. 
This corresponds to projecting the raster of feature points onto 
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the X-axis at some angle. 

The program (PROJ X SL) carries out this projection 
procedure on an array of feature points X, projecting at an 
angle SL. The main program calling PROJ uses 100 values of 
SL. For 5 values, the feature points obtained from horizontal 
scans (parallel to the x-axis) are used. The values of SL in this 
case correspond to orientations between + 45 degrees with the 
vertical (the y-axis). The remaining 50 projections use the 
feature points from the vertical scans, and involve angles 
between + 45 degrees with the horizontal. The values histogramed 
are not simply numbers of points, but sums of integer weights 
attached to the various feature points. These integers have 
values from 1 to 4, and depend monotonically on the value of 
R" - Min(R, R'), the first feature point value, for a particular 
point. Thus, for a particular band, it is not simply the quantity 
of feature points within it that is considered, but something 
approximately equal to the sum of the values of R" - Min(R, R') 
for points within it. This gives extra weight to distinct lines, 
and may be shown to approximate Q* in some sense. The value 
returned by (PROJ X SL) is the set of x-intercepts of centerlines 
of bands of slope SL which have a large amount of line evidence 
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in them. The algorithm employed by PROJ is diagrammed in 
Figure 4. 11. 

The set of outputs of (PROJ X SL) must be analyzed to 
extract the actual lines in the scene. One problem which must 
be taken into account arises from the fact that a particular set 
of feature points may give rise to an intercept value in the 
output of several applications of PROJ, for successive values 
of SL. This is a result of the fact that a given set of feature 
points which fall in a line along the visual field are contained 
within several rectangles of almost identical slopes. Another 
problem is that the output of PROJ contains a large number of 
false -positive values due to spurious effects described 
previously. 

The program LINES, which calls PROJ for various 
values of the arguments, takes these factors into account by a 
complex process of: 

1) retrieving the feature points in sets from areas 
suggested by the output of PROJ, 

2) eliminating sets which do not actually constitute lines, 

3) eliminating spurious points from sets which do represent 
lines, 
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4) sorting by the apparent strength of lines, 

5) eliminating redundant sets of points by considering the 
set in the order imposed by step 4 and eliminating sets 
which are highly redundant with (more distinct) sets 
which had previously been considered, 

6) fitting lines to the resulting sets of points. 

The output of (LINES X) is a set of lines retrieved from the set 
of feature points X. LINES is applied once to the feature points 
from the vertical scan; and once to the feature points from the 
horizontal scan. The result of applying LINES to the sets of 
feature points illustrated in Figure 4. 10 is shown in Figure 4. 12. 

V. LINKING LINES TO FORM PARTIAL FIGURES 

Having located a certain number of lines in a scene by 
the methods of the previous section, a logical next step is to 
join them together and form figures. This may be viewed as a 
terminal step, provided all lines have been located by the 
projection procedure. This would not, in general, be the case; 
and the joining procedure described in this section is actually 
a first step in the location of the remaining lines of the figure. 
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OUTPUT OF LINES, APPLIED TO THE FEATURE POINT 
RASTERS IN FIGURE 4. 10 

Figure 4.12 
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We shall assume that the output of LINES, when applied 
to a set of feature points, does not include all the lines of 
interest in the original scene. This is, in fact, the case for 3 
out of 4 scenes in Figure 4. 12. It is a reasonable hypothesis 
that the lines found, together with some assumptions about the 
nature of the scenes examined, provide some clues as to the 
locations of the remaining lines. It appears not to be the lines 
individuadly which suggest the locations of other lines, but 
rather collections of lines which form a figure which is 
incomplete relative to a model of how the actual figures appear. 
An example is shown in Figure 4. 13, 




THE SOLID LINES HAVE BEEN IDENTIFIED, 
THE DOTTED LINE IS SUGGESTED BY THEM, 

Figure 4. 13 
It would be desirable to join the lines into groups which would 
suggest the possible location of other lines. A line verifier 
could then be applied to the locations suggested. This 
procedure requires that all locations of missing lines ultimately 
be proposed. Otherwise, some lines would never be found by 
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either procedure. It is thus important that the whole mechanism 
for line proposing be liberal in proposing lines. Any excess lines 
proposed will almost certainly be rejected by a suitably powerful 
verifier. 

Prior to the application of the linking procedure, a 
merging procedure is applied to the lines. This is necessary 
to eliminate redundant lines from two sources: 

1) Lines which are at approximately at a 45-degree angle 
with the vertical, and appear in the output of LINES 
applied to both the vertical and the horizontal scans. 

2) Single lines which appear, for one reason or another, 
as a pair of contiguous lines in the output of LINES. 
The criterion for merging lines is given in Figure 4. 14. 



d<10 and 0<1O' 




|d| + |e|+|d + e |<50 

CRITERIA FOR MERGING LINES 

Figure 4. 14 
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If two lines are to be merged by this criterion, the pair is 
replaced by a single line which best fits the endpoints of the 
two original lines. 

The joining procedure consists in applying directed 
links from one line to another. There are four possible types: 

1) Regular links, denoted by the symbol T. If any part of 
line B lies within a neighborhood of radius 10 units 
around A, then A is given a directed T-link to B. 

This is illustrated in Figure 4. 15. 

2) Extension links, denoted by the symbol E. If any part 
of line B lies in a particular region near the end of A, 
or if the end of line B lies within another more distant 
region extending away from the end of line A, then an 
E-link from A to B is established. These regions are 
illustrated in Figure 4. 15. The motivation for this type 
of link is that possibly a segment of A extending to B, or 
a longer segment extending to the end of B, has been 
omitted. The reason for postulating a rather long 
extension to the end of another line is that a line's 

end, lining up with a given line, provides strong 
evidence that the original line may actually extend to 
that end. 
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Figure 4.15 

3) Passive regular links, denoted by PT. If line B links 
to line A by a T-link, but A does not link to B by eitiier 
type of link, then a PT-link is established from A to B. 

4) Passive extension link, denoted by PE. This is similar 
to a PT-link, but is applied if B links to A by an E-link. 
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An illustration of the links applied to one of tlie sets of 
lines from Figure 4. 12 is given in Figure 4. 16. It should be 
emphasized that, in accordance with the necessary liberal 
policy underlying the proposing of lines, the lines are over-linked. 
These links are not intended to be those applied when all the lines 
have been located. 

VI. HEURISTIC LINE PROPOSING 

This section describes the program (PROPOS X), which 
proposes additional lines from a possibly incomplete set X of 
lines from a scene. Each line is considered in turn, and 
additional lines suggested by it, and by lines linked to it, are 
proposed. A verification program, to be explained in the last 
section, is applied to each proposed line. If the verifier claims 
that the line actually exists, it is added to the list of lines, 
together with links between it and other lines; and it may be 
used subsequently in proposing further lines. When no more 
lines can be proposed from the (possibly augmented) set of lines, 
PROPOS returns the entire set. 

The proposing procedure is based on the premise that if 
a line is missing from a set S, then at least one line having a 
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Figure 4. 16 
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common end-point with the missing line is in the set X, or can 
be successfully proposed from X. This is illustrated in 
Figure 4. 17. ' It is further assumed that vertices have no more 




(missing)'^ 



IT IS ASSUMED THAT IF A LINE OF THE 
ORIGINAL FIGURE WAS NOT AMONG THOSE 
OUTPUT BY LINES. THEN AT LEAST ONE 
LINE CONNECTING TO IT WAS. 

Figure 4. 17 

than three or possibly four lines radiating from them. It 

follows that lines should be proposed from the ends of lines 

already found, provided that there are not already many lines 

incident with a particular end. 

It is in general not entirely clear where a line generated 

by LINES actually terminates. This is partially due to the 

rough "mesh" of the scanning grid, and partly due to the fact 

that the feature point generator gets confused in the neighborhood 

of a vertex. In order to propose lines from the ends of known 
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lines, it is necesssiry to make a reasonable estimate as to 
exactly where the ends lie. The first step in proposing lines 
from a given line is to dispose of the ends in the following 
manner: 

1) If two lines intersect within ten units of the end of a given 
line, the point on the line (possibly extended) closest to 
the intersection is taken as the end or "terminus". 

2) If a terminus is not determinable by criterion 1, but a 
line intersects the given line within 25 units of the end, 
this intersection is taken as the terminus. 

3) If no terminus can be found by 1 or 2, that end of the 
line is said to be of "indeterminate terminus. " 

The determination of termini is diagrammed in Figure 4. 18. 

Another assumption made by PROPOS, which is 
independent of the first, is that the opposite sides of faces of 
the figures are approximately parallel. If it is desired to 
propose a line extending, e. g. , downward from the left 
terminus of a horizontal line, the direction of the proposed 
line is then the same as that of another "secondary" line 
extending downward from the given line. In particular, this 
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THE "TERMINI" OF THE HORIZONTAL 
LINES ARE AS INDICATED. 
LINES PROPOSED FROM THE HORIZONTAL 
LINES EXTEND FROM THE TERMINI. 



Figure 4. 18 
"secondary" line should be the first line "to the right" of the 
left terminus, as that is the obvious candidate for an opposite 
side to the proposed one. This determination of these secondary 
lines, termed "directors" is diagrammed in Figure 4. 19. 
Having determined the termini and directors for a 
particular line, the procedure for proposing lines from a 
particular end is governed by four cases: 
1) The end of the line is of undetermined terminus; 
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Figure 4. 19 

2) Only one line intersects with the given line at the 
selected end; 

3) Two lines lying on the same side of the given line 
intersect the selected end; or 

4) Something else occurs at the selected end. 

In the first case lines are proposed from either side of 
the original line for which a director has been found. Three 
parallel lines are proposed, 10 units apart, parallel to the 
director, and with the end of the middle line coinciding with 
the indicated end of the given line. The other ends of the 
proposed lines are determined in this case, and in the cases 
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which follow, by making the proposed line no longer than the 
director, and truncating it so that it does not intersect any other 
lines in the set. In case 2), lines are proposed on the opposite 
side from the intersecting line, provided a director exists on 
that side. Three lines are proposed ten degrees apart, all 
extending from the terminus. Their lengths are determined 
as above, and the middle line is again parallel to the director. 
If no lines are found on the side away from the intersecting line, 
then lines are similarly proposed on the same side as the 
intersecting line, subject to the additional provision that the 
proposed lines all lie between the intersecting line and the 
director. If this were not the case, it would be impossible that 
the proposed line and the director be opposite sides of a face. 
In case 3), lines are proposed similarly to the first set 
described for case 2). In case 4), no lines are proposed. 
These cases are diagrammed in Figure 4. 2 0. 

In summary, PROPOS goes down the list of linked lines 
proposing in the manner described above, adding lines which it 
finds to the end of the list. This procedure is applied to the 
resulting list, and so on until no lines on the current list result 
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PROPOSING OF LINES FROM AN END OF A GIVEN ONE 
IS GOVERNED BY FOUR CASES. 

Figure 4.20 
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in the proposing of aiay further lines. At this point, the final 
set of lines is returned by the program. The process of 
proposing lines is diagrammed in Figure 4. 21. 

VII. A LINE VERIFIER 

Line verification consists in applying the function 

Q*(j") as given by {3. 13) and (3. 29), or some variant of it, to 

the intensities J-, or some vsilues derived from them, lying 

in a region suggested by the line proposer. The region is 

assumed to be divided into n square or nearly square sub- 

regions, and Q* or its variant is computed from the values 

R. 1, . . . , R. , a* , . . . , etc. , according to Formulas 
J^ 1 j, n J, 1 

(3. 13) and (3. 29), or formulas derived from them. In most of 

the remainder of this section we shall be concerned with a 

generalization of Q*( J-; ) which has been suggested by the 

discussion at the end of chapter three. This generalization 

consists in assuming that the values of R. , and R'. , are 

3^^ J.k 



given by: 



R. , = Min(R. R. , c)» (4. 1) 

J.k J,k, 1 J,k, 6 

R' , = Min(R' , . .. . , R! , ), (4.2) 

3^k ],k, 1 J,k, 5 
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Figure 4.21 
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where the R. , 's and R' 's are computed from the 25 

J^k, t j,k, u 

intensities of the k-th sub-region scan segment according to 
(3. 25), and from the paradigm intensity profiles illustrated in 
Figure 4. 7. According to the discussion in Section III, the 
values in the feature point rasters are derived from the 
values given by (4. 1) and (4. 2), and the present line verifier 
draws upon exactly these values. In this section we shall 
review tiie relevant theory in detail, describe the present line 
verifier, and suggest possible improvements and generalizations 
of it. 

The present line verifier, in computing a variant of 
Q*(J-,) for a region, obtains the values of, e. g. , R for 
CL(j) of (3. 29) from the feature point rasters such as are 
partially depicted in P'igure 4. 10. In principle, if the region 
encloses n 25 point sub-region scan segments of the original 
scans, then the values R. etc. , should be computed from 

exactly these scan segments. These values were computed 
for all possible sub-region scan segments, in the generalized 
manner given by (4. 1) and (4. 2), by the feature point generation 
procedure. However, most of the values of the R's and R"s 
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were discarded during the compactification process. In fact it 

was only locally maximal values of E, given by: 

E. , = R'.' , - Min(R. , ,R'. ) (4.3) 

where R. , and R' are given by (4. 1) and (4. 2), that were 
3'k j,k 

retained. However, it turns out that we may obtain a 
reasonable estimate of the value of E. , for the intensities of 
a particular sub-region scan segment along a particular scan 
from one of these maxima nearby along the same scan. In 
particular, if E. , is a desired value of E for a sub -region 

A 

scan segment along some scan, then let E be the value of 

3» k 

E which is locally maximal along the same scan and is computed 
from a sub -region scan segment closest to the one in question. 
We assume that there exists a maximum within something like 
15 units of the segment in question, else we assign E. the 

A 

value 0. If E- , so defined actually exists, then the following 

relationship holds: 

E , « E. , - ax^ (4. 4) 

3>k 3,k 

where a is some constant, ajid x is the distance between the 

A 

sub-region scan segments from which E. , and E. , were 
^ ^ 3,k j,k 

computed. This relationship follows from the behavior of the 
E curve in the neighborhood of a maximum, which may be 
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observed in Figures 3. 9 and 3. 10. It will be seen later that 
the n values of E. , are all that are necessary to compute the 
present variant of Q* used in the line verifier. 

This present variant of Q* omits the second and third 
terms in the exponent of, e. g. , the value of CL as given by 
(3. 29), and thus does not need the values of a* , , a'.*, and 
a'.'* . The middle term may be ignored because it turns out to 
be always very small relative to the other two. To see this, 
recall that line verification in the present set of programs is 
applied only to lines, or possible lines, which escaped 
detection by the LINES program. They are thus lines which 
have a net relative amplitude whose magnitude is small relative 
to the average magnitude of relative amplitudes for lines in 
general. One may observe that the denominator of the middle 

term is dominated by its third term, so the term is approximately 

2 2 

( Sa'' i^/n)/p . This is approximately the square of the net 

amplitude divided by its variance. Since the net amplitude 
in situations under consideration is small relative to the 
average magnitude of the net amplitudes, and the average net 
amplitude over lines in general is zero; then the net amplitude 
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is small relative to its standard deviation. The second term of 
(3, 29) is thus in this case very small, being approximately the 
square of the quotient of the net amplitude with the standard 
deviation. 

The omission of the third term of the exponent of (3. 29) 
was based on the empirical observation that for the majority 
of the scenes analyzed, this value seemed not to be related in 
a reliable way to the existence of lines. A series of experiments 
was begun to properly test whether any such relation existed. 
The results appeared promising but not so far conclusive. 
Consequently the third term was omitted pending further 
investigation. 

The resulting variant of Q*(J •) is quite similar to the one 
discussed at the end of the previous chapter. It is given by (3. 13), 
where the various values, e. g. , of CL are given by (3. 2 9) but 
with only the first term in the exponent. This was the case in 
the situation described in the end of chapter three. Also, this 
time we expect a scan of 500 points to intersect with about five 
lines in the visual field, so the relation between P(CL), P(CE) 
and P(CH) remains approximately the same as in the situation 
at the end of chapter three. Finally, we may as before 
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assume that one of the terms in the numerator is considerably- 
larger than the other. Thus by an argument similar to that 
used in deriving (3, 41) and (3. 43) we arrive at the function 
Q'**(J^) defined by: 
Q'**(j") = 

n 

VMaxCSL, (j)-SH (j), SE, (j)-SH(j)) + K, (4. 5) 

1^1 ^ k k 

where, e. g. , SL, (j) is like SL(j) given by (3. 37) with subscript 
k in place of 1. As before, this function is approximately 
monotone in Q*(J4) and hence thresholding it provides an 
approximately sub -optimal regional decision predicate. One 
may see from (3. 37), (3.38), (3.39), (3.23), (3.24), (3.25) 
and a certain amount of algebra that: 

and 

SE, (j) - SH, (j) = R'.' , - R'. , . 
k'-" k'"" j,k ],k 

Using (4. 3) we may rewrite (4. 5) as: 

n 
Q'**(j") = Ve +K, (4.6) 

J k=l '•' 

with the values of E. , obtainable via (4. 4) from values 

A 

E. , , which are available in the feature point raster. In 
h k 
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summary we have: 

Q ->!<*( jj) = 2^ Ej k - a^k + K, (4. 7) 

E. ,*0 

recalling that if a suitably close feature point for obtaining a 

A 

value E. , did not exist, then the corresponding E . was 
assigned the value zero. 

In practice it was found to give better results to 
deviate slightly from a strict thresholding of (4. 7) in the 
verification of lines. Prior to applying (4. 7) to a region it was 
observed which of the E. , were given nonzero values. In the 
case that the locations of the nonzero E. 's were sparse or 
concentrated in one place in the region, the thresholding 
procedure was omitted, and a negative answer was returned 
as to the existence of a line. This is approximately the same 
as observing the pattern of the locations of feature points within 
the region, and denying the existence of a line if they are all 
clumped together or are sparsely scattered within the region. 
These two situations are usually due to the existence of feature 
points from lines that cross the region at non -negligible angles. 
Another variation from (4. 7) which was found to be more 
convenient, and in some cases more accurate, was to 
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A 2 

threshold the sums of the E- , 's and the sums of the x, 's 

separately. Finally, instead of taking x, to be the deviation of 

A 

the E. , feature point from the center of the region, it was taken 

A 

to be the deviation of the E. , feature point from a best -fit line 

A 

to the various E , feature points. 

The line verifier (VERIFY PI P2) takes as arguments 
the end -points PI and P2 of the proposed line and outputs 
either NIL, if the line is thought not to exist by the above 
criterion; or the end -points of the best fit line to the feature 
points if the line is considered to exist. 

In observing the line verifier in action in Figure 4. 21, 
occasional failures are apparent. Although no lines are claimed 
to exist which do not exist in the figure, several actually 
existing in the figure were proposed but claimed, on the basis 
of the feature point raster information, not to exist by the 
verifier. An examination of the feature point raster indicates 
the presence of feature points in these regions. The problem 
was principally that the region intersected with too few scans, on 
the order of three to five; giving too few values for the threshold 
function (4. 7) to work with. Evidently a finer scan would 
eliminate this problem, as well as some other approaches to 
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its improvement which are about to be discussed. 

A major source of improvement for the line verifier is 
for it to obtain a new set of intensities within the region to which 
it is applied, at considerably higher density than that of the 
original scan. This would eliminate the difficulty described in 
the previous paragraph, would allow more accurate values of 
the R's, E's or whatever, to be obtained, and would eliminate 
the necessity of the approximation to the E • i^'s in (4. 4). 
Another improvement would result from the utilization of the 
third term of (3. 29) and similar expressions. As was stated 
previously, investigations in this area are still under way. 
It seems plausible that the model in Section IV of chapter three 
may have to be modified in assumptions 4) and 5) to admit of a 
regular linear variation of "idealized" relative amplitude along 
a noise-free ridge-like or cliff-like noise-free sample. 
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CHAPTER 5 
RELATED WORK 

I. OTHER APPROACHES TO LINE FINDING 

An early line -finding program was written by Roberts 
(Roberts). This program analyzed photographs, of good 
contrast and focus, of scenes whose content was principally 
defined by straight edge -lines. Feature points were obtained 
from a predicate on four adjacent intensities of a 2 by 2 square 
on the visual field. Feature points were organized into lines 
by a sort of line -following procedure. Higher level heuristics 
handled the resulting lines. 

A comparison may be made between the behavior of 
Roberts' four-point feature point predicate, and the present 
feature point gathering procedure. This analysis is a 
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generalization of a procedure due to Binford and Herskovits 
(Herskovits). 

An upper bound on the performance of the former may- 
be inferred from theoretical considerations. Data on the actual 
performance is not available. The feature point predicate is 
applied to intensities X. x.^^ ., x. , andx^^^^^^^ at four 
adjacent points in a square pattern on a 256 by 256 point grid. 
The thresholded value is: 

where 



Assume that the noise is gaussian, and that the noise level, as 
measured by the standard error about a mean of values of j_ 
taken at a single point under constant lighting conditions, is s 
for intensities in the vicinity of the average of the four y's in 
question. It is not difficult to show that z^ as a function of the 

random variables y^j, y^+ij. ^i, j+1 ^^^ ^i+l, j+i ^^^ ^ 
median of about 2s, and an upper interquartile point value of 
about 3s. In the context of the set of figures we are considering, 
one would expect about one feature point per 100 quadruples of 
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intensity. Thus for about 50% false positives, one would wish 
to set the threshold on z^ high enough that z^ falls above threshold 
by chance once in 100 times. For a normal distribution, this is 
about 2. 5 standard deviations, which corresponds to 3 times the 
semi-interquartile range. Thus z should cut off at about 5s. 
Now it is not difficult to see what values of z would result for 
various lines and edges of amplitude a oriented in various ways 
with respect to the grid of squares. In Figure 5. 1 we have 
computed such values for 12 cases. In the diagram, we are 
assuming that the dimensions of the blurring function are such 
as to give the line and edge profiles of the width shown. This 
width appears to be about an optimum compromise. If 
considerably lower then more edges would have a relatively 
high z -value, but more lines would have a zero z -value. As 
the width increases, both lines and edges tend toward having a 
zero z-value. From the diagram it may be estimated that the 
average value of z for edges is about . 8a, and for lines about . 4a. 
Thus the amplitudes of lines and edges respectively which are at 
threshold sire about 6. 3s and 12. 6s. This may be compared 
with the values 1. 2s and 2s obtained for Q* at the end of 
chapter three. The difference is due to two factors. First, 
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the Roberts predicate has a support of only four points, whereas 
the Q* tested had a support of 25 points. One would expect the 
signal to noise ratios to differ by a factor of\/25/4 = 2. 5. 
The actual difference is about 5, and the remaining factor of 
two is probably due to the fact that the Roberts predicate is 
isotropic, whereas Q* is directional. 

The Topologist, based on the work of Binford, Sussman 
(Sussman) and Herskovits (Herskovits), represents another 
approach to the analysis of simple scenes. In this case, the 
scene is divided into a set of small square regions which are 
individually tested for homogeneity. Adjacent homogeneous 
regions are clustered into maximal homogeneous regions, which 
are presumably the areas of the scene which lie between edge 
lines. This procedure has the properties that edge lines need 
not be straight, and that arbitrarily complex predicates for 
homogeneity may be used. 

Recently, a method of spatial frequency filtering and 
matching has been applied to the problem of location of feature 
points (Hueckel). This procedure is roughly a two dimensional 
version of the feature point analysis presented here. For a 
circular neighborhood, an estimate is given of the directional 
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and amplitude parameters of a possible edge through the 
neighborhood. This procedure seems to be based on a rather 
simple model of the nature of intensity profiles at edges. Also, 
it seems to be relatively insensitive to the presence of lines. 

An approach bearing some mathematical similarities to 
the feature point detection procedure reported here, was 
employed by Saunders (Saunders) in an analysis of filmed 
images of scope tracings. In this case it was necessary to 
locate the intersections with parallel scan lines of the image of 
a narrow trace along the film. 

Worthy of mention here, though not specifically related 
to line finding, is the work in statistical decision theory of 
Chow (Chow). The theorem in the first chapter is essentially 
an adaptation and generalization of Chow's results in the theory 
of optimal error reject trade-off. 



II. RELATION OF THE PRESENT WORK TO THE 

DEVELOPMENT OF A VISUALLY ORIENTED REAL- 
TIME OBJECT MANIPULATOR 



The program here described is part of a larger effort to 
develop the visual-perception aspects of a real-time object 
manipulator. (MAC Progress Report 1968) For the present, 
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effort has centered on programs to obtain descriptions of scenes 
composed of cubes auid other prismatic solids. By a description 
of an object is meant a list of the spatial co-ordinates of its 
vertices, together with a list of pairs of vertices defining edges 
of the object. By a scene description is meant a list of object 
descriptions, one for each object in the scene. 

There is an obvious disparity between the output of the 
present program and a corresponding scene description. The 
former is, or can easily be reduced to, a set of vertices and a 
set of vertex pairs representing edge lines. A description 
requires that these vertices and vertex pairs be grouped into 
sets corresponding to the various objects in the scene. One 
may regard the output of the current program as a set of lines 
dividing the field into regions. The problem then becomes one 
of grouping regions which correspond to faces of a single object, 
and matching regions which may be parts of a single face which 
has an occluding object in front of it which divides it into two 
regions. This problem has been investigated in detail, and a 
program called SEE (Guzman) performs the described grouping 
and matching to reduce the output of the present program into a 
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proper two dimensional scene description. 

Several approaches to the problem of determining the 
locations of objects in three dimensions have been investigated 
in greater or lesser degrees of detail. One suggestion involves 
the use of two light sources located close to the point of view of 
the optical input device. One would be a point source, whose 
intensity falls off as the square of the distance from the point of 
view. The other would be a source placed "at infinity" by means 
of lenses, whose intensity would fall off negligibly with intensity. 
If illumination is rapidly alternated between these two sources, 
and intensities obtained for a particular point under both 
conditions, the relation between the two intensities would yield 
the distance from the point of view to the point in the field. 
Another approach to determining the distance from the point of 
view to a point in the scene involves the use of optical focusing. 
(Horn 1968) By this procedure, a point in the scene is brought 
into perfect focus and the position of the lens of the optical input 
device is transmitted to the computer. After a suitable 
calibration procedure, the absolute distances of objects may thus 
be obtained automatically. A third approach (Horn) involves the 
use of intensity gradient to infer the shape of the contour of a 
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curved surface. By this technique, relative distance information 
may be obtained for points on curved surfaces. A final approach 
worked out by the present author, involving an analysis of the 
scene from two different points of view, relates directly to the 
current work. It is assumed that the present program has 
examined a suitable scene from two different points in space, 
and that the resulting two sets of lines have been processed by 
SEE. Central to this approach is a subroutine called WHERE 
which is based on results in projective geometry, and is 
described in a recent Project MAC Artificial Intelligence Memo 
(Minsky). This program takes as input the two dimensional 
co-ordinates of a point from each scene. In many cases, WHERE 
can determine that the two points are not images of the same 
point in space. If the points are from the same point in space, 
WHERE outputs the three-dimensional co-ordinates of that point. 
One might apply this program in a straightforward manner to a 
point in one scene together successively with all points of the 
other. This would greatly reduce the possible "matches" of 
features in the two scenes, but would, in general, not suffice to 
determine the three -dimensionEil locations of all the vertices. 
The problem of resolving the remaining ambiguities, as well as 
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developing a more efficient point -matching procedure seems to 

be quite complex. It seems likely that the organization that 

SEE imposes in the lines would provide a basis for using 

contextual clues to match the vertices. The problem is currently 

under investigation. 

A final problem worthy of mention is that of the 

determination of the stability of a configuration of prismatic 

solids, given a three-dimensional scene description of it. 

(Blum) The relevance of this problem derives from the fact 

that it is one of the goals of the object manipulator project to 

be able to construct structures from rectangular blocks. In 

order to direct the object manipulation device, a program 

must first determine the order in which the blocks are to be 

set down. It may be possible to construct a sub -configuration 

of the desired configuration which has the property that adding 

any additional block results in an unstable configuration. To 

forestall such a possibility, it is necessary to be able to 

detect instability in searching for an appropriate order in which 

to build the structure. 
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